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I. INTRODUCTION 


A. BACKGROUND 

Competition requirements for defense procurement under 
conditions that do not need a waiver require two or more 
Seweense contractors to compete for contract award on a 
solicitation [4: p.2]. The defense agency selects a winner or 
A ers of the competition and the other contractors are 
Memecessftul offerors for that particular competition. When 
price alone is not the only criteria used to determine 
contract award the unsuccessful offerors may be very 
interested to understand why they did not win the contract 
award. By requesting a debriefing from the agency that is 
responsible for the procurement, unsuccessful offerors can 
obtain the information they need to determine why they lost 
the procurement competition. The debriefing is a meeting 
between representatives oof the procuring agency and 
unsuccessful company where agency representatives will explain 
Mabe company why they did not win the contract award. 

Unsuccessful offerors do not receive formal debriefings 
for all defense procurement actions. Rather only one of the 
two methods for federal procurement warrants a formal 
Seemeefing [7: p.15-41]. The two possible methods of 


procuring goods and services in federal procurement are sealed 


bidding and negotiated procurement. The first method, sealed 
bidding is to be used when (a) there is adequate time, (b) 
contract award will be made based on price and price related 
factors, (c) no discussions with offerors are needed, and (d) 
there is an expectation of receiving bids from more than one 
offeror [4: p.1]. The second method, competitive negotiation, 
is to be used if the procurement doesn’t meet the criteria of 
sealed bidding. 

The reason for contract award using sealed bidding is easy 
to determine. The lowest priced bid from the offeror who is 
both responsible and responsive wins the contract award 
[97n p723 ]S Thus, unsuccessful offerors generally know 
exactly why they didn't win the contract award and it is 
normally because they didn't have the lowest priced bid. 

For competitive negotiations the reason for not winning 
contract award may not be so easy to determine. With the 
advent of best value contracting, discovering the reasons for 
not winning contract award may be further clouded from view. 
This is because in this type of contracting, contract award is 
made based on cost plus one or more other factors. Examples 
of other factors could be technical solution, management 
experience, past performance, and risk. In addition, the 
factors are weighted in accordance with their importance to 
the contract award. Once an evaluation is made and a winner 
is selected, the unsuccessful offerors are notified that they 


were not selected for contract award. With a multitude or 


selection criteria and different weights used for each 
criteria, unsuccessful offerors may have little idea why they 
lost the contract award. It is no longer a case of realizing 
that his cost was too high. It may be that his technical 
solution, or management experience or some other factor or 
combination of factors, when evaluated in accordance with the 
evaluation criteria, were not as good as the winner's. The 
unsuccessful offeror can request a debriefing from the agency 
that is responsible for the procurement to find out what the 
actual reasons are for him not being selected for contract 
award. 

With the recent cuts in military procurement resulting 
from the end of the cold war debriefings have increased in 
importance for defense contractors as they fight for a piece 
of the remaining procurement pie [12]. By understanding why 
they lost one award, defense contractors can use the 
information they obtained from a debriefing to increase their 
competitive position for the next procurement. 

Debriefings also have the potential to help the Department 
of Defense (DOD). From it's peak in fiscal year (FY) 1985 the 
defense procurement budget will decrease approximately 65% to 
Ito Jected $43.3 billion for FY 1995 [8: p.8]. Budget 
constraints caused the Army alone to cut 57 procurement 
meegeams and scale back on 77 others in FY 1993 [16: p.10]. 
The dollars left for procurement must be wisely and 


efficiently spent. Debriefings can be used to increase the 


competitiveness of future procurements and thus allow DOD to 
get more for each procurement dollar. 

Debriefing unsuccessful offerors is not something new. 
Debriefings were mentioned as far back as 1972 in the Report 
of The Commission on Government Procurement, published in 
December 1972. Additionally, debriefing unsuccessful offerors 
was required by the Defense Acquisition Regulation, in force 
before the advent of the Federal Acquisition Regulation (FAR) 
in 1984 [(6: p.D.15.10-1]. The FAR states that debriefing of 
unsuccessful offerors that request a debriefing is required 


when a contract is awarded on the basis of other than price 


alone [7: p.15-41]. The FAR provides some general guidance 
for the debriefings. This is as far as federal laws and 
regulations go in covering the debriefing process. Army 


regulations add little to what the federal regulations 
stipulate. Details of who debriefs, when and where the 
debriefing occurs, and the content of the debriefing are left 
to a command’s standard operating procedures or contracting 


officers to decide. 


B. OBJECTIVES 

The objective of this Thesis is to examine the Army 
Material Command’s process of debriefing unsuccessful offerors 
in contracts awarded on a basis of other than price alone. 


The research is focused on what the current debriefing process 





is, what problems there are with the current process, and what 


can be done to improve the process. 


C. RESEARCH QUESTIONS 

The following research questions will be answered by the 
Thesis. 

Primary: What are the key problems and issues associated 
with the Army Material Command's process of debriefing 
unsuccessful offerors and what can be done to improve the 
process? 

Subsidiary: 

1. What is the current debriefing process used by Army 
Material Command? 

2m What are the problems associated with the Army 
Material Command’s current debriefing process from Army 
Material Command’s perspective? 

3. What are the problems associated with Army Material 
Command’s current debriefing process from the unsuccessful 
offerors perspective? 

4. How can the debriefing process be modified to help 


Army Material Command and the unsuccessful offerors? 


D. SCOPE, LIMITATIONS, AND ASSUMPTIONS 
1. Scope. This Thesis focuses on the debriefing of 
unsuccessful offerors in Army Material Command (AMC). IE 


covers all contracts awarded on a basis of other than price 


alone. Five of the six major subordinate commodity commands 
are studied. They include the Armament, Munitions and 
Chemical Command, the Aviation and Troop Command, the 
Communications-Electronics Command, the Missile Command, and 
the Tank-Automotive Command. The Chemical and Biological 
Defense Command is not looked at as it recently reorganized 
and just became part of AMC in 1993. Surveys and interviews 
on the current process and how to improve it are directed to 
Army Material Command contracting officials and defense 
contractor contracting representatives who contract with AMC. 

2. Limitations. The following limitations exist in the 
thesis. 

a. Legislation on streamlining defense acquisition, 
which includes a portion on debriefing unsuccessful offerors 
is currently being discussed in Congress. Uncertainty as to 
exactly what that legislation will be will exist until it is 
passed, if it is passed. 

b. Not all contracting officers within Army Material 
Command and defense contractors that work with Army Material 
Command were contacted. Additionally several were contacted 
but did not respond to the survey or interview request. 
Therefore there may be information useful to the thesis that 
was not collected and thus not available for presentation and 
analysis. 

3. Assumptions. The following assumptions were made in 


this thesis. 





a. The reader of the thesis has a general 
understanding of Government contract administration. 
lo»: Current regulations concerning debriefing of 


unsuccessful offerors will remain in effect. 


E. DEFINITIONS AND ABBREVIATIONS 

1. AMC: Army Material Command. A major command in the 
Army. AMC is responsible for developing, acquiring and 
sustaining the material the Army needs to fight its wars. It 
has many subordinate commands that perform specific missions. 

2. AMCCOM: Armament, Munitions and Chemical Command. A 
major subordinate command of AMC. AMCCOM is responsible for 
the development, production and fielding of Army artillery, 
mortars, rifles, rocket launchers, and aircraft and tank 
armament. It is also responsible for acquiring all Army 
ammunition. 

3. ATCOM: Aviation and Troop Command. A major subordinate 
command of AMC. ATCOM is responsible for overall acquisition 
of all Army aviation assets and individual equipment for 
soldiers, such as clothing, food and facilities. 

4. CBDCOM: Chemical and Biological Defense Command. A 
major subordinate command of AMC. It is responsible for 
research, development, and production of biological and 
chemical defense items. 

Er CECOM: Communication-Electronics Command. A major 


subordinate command of AMC. CECOM is responsible for the 


overall acquisition of Army communications and electronic 
equipment. 

6. MICOM: Missile Command. A major subordinate command of 
AMC. MICOM acquires all Army tactical missiles and rockets. 

7. MSC: Major subordinate command. 

8. SSA: Source Selection Authority. 

9. SSAC: Source Selection Advisory Council. 

10. SSEB: Source Selection Evaluation Board. 

11. TACOM: Tank-Automotive Command. A major subordinate 
command of AMC. TACOM is responsible for the overall 
acquisition of Army tanks, automotive ground vehicles, 


construction equipment and material handling equipment. 


F. ORGANIZATION OF THE STUDY 

Chapter II (background) discusses the following four major 
areas. The organization of AMC and its MSCs; where debriefing 
fits in the acquisition cycle; why debrief unsuccessful 
offerors and the laws and regulations that pertain to 
debriefing unsuccessful offerors. 

Chapter III (research methodology) describes the 
rationale behind the surveys and details the interviews. 

Chapter IV (data presentation and analysis) presents a 
Summary of the surveys and analysis of the results, and 
presents and analyzes the data collected from the interviews. 

Chapter V (conclusions and recommendations) discusses the 


conclusions that are made from the data collécted and makes 


recommendations for improving the current debriefing process. 
This chapter also provides areas that are recommended for 


further study. 


II. BACKGROUND 


A. GENERAL 

This chapter will address the areas that the reader must 
be familiar with in order to understand the specific 
discussion on debriefing held in the later chapters. It will 
discuss the laws and regulations that pertain to debriefing, 
how debriefing fits in the source selection process, and the 
reasons why debriefings are conducted. This chapter will 
also provide a description of AMC and show where contracting 


officers fit in their organization. 


B. LAWS AND REGULATIONS ON DEBRIEFING 
1. Laws 

Title 10 of the United States Code contains the laws 
which pertain to negotiated procurement for the military. 
Currently there is no legislation contained in Title 10 that 
pertains to debriefing of unsuccessful offerors. Title Ti 
mentions only that unsuccessful offerors be notified promptly 
once contract award has been made. This was added to Title 10 
in 1988. [23: sec.2305] This is not to say that Congress is 
not interested in debriefing or hasn't discussed it. Rather, 
the recommendation to pass legislation concerning debriefing 


has been brought up several times. It is discussed in the 


Report of the Commission on Federal Procurement, published in 
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December 1972, surfaced again in 1991 when both houses of 
Congress proposed legislation that covered debriefing, and 
appeared again in 1993 as part of the Section 800 Panel Report 
that went to Congress. 


a. Report of the Commission on Government 
Procurement 


The Report of the Commission on Government 

Procurement was conducted in 1972 by a commission appointed by 
Congress and the Executive Branch. It consisted of two 
members each from the Senate, House of Representatives, and 
Executive Branch, one member from the Comptroller General, and 
five members from the public sector. It was commissioned in 
response to a concern over the manner in which the federal 
procurement process operated and over the deficiencies in the 
system [18: p.1]. It was a full scale study of the entire 
federal procurement process and recommended to Congress 
methods to increase the economy, efficiency, and effectiveness 
of the procurement process. One of those recommendations was 
on post-award policy and pertained to debriefing of 
unsuccessful offerors. The recommendation was that: 

When competitive procedures that do not involve formal 

advertising are utilized, establish that agencies shall 

Upon written request of an unsuccessful proposer, 

effectively communicate the reasons for selecting a 

proposal other than his own. [18: p.25] 
The report’s reasons for the recommendation was that there 


were no laws requiring communication with unsuccessful 


offerors to explain why their proposals were not as 
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advantageous to the Government as the winning offeror's. 
Letting an unsuccessful offeror know why his proposal was not 
as strong as the winner's allowed the offeror to improve and 
be more competitive in the future. Debriefing, the report 
explained, improves the confidence of offerors as they realize 
the procurement rules are being followed and applied fairly. 
Unsuccessful offerors thought they should be given information 
to understand the value of their proposals [18: p.25]. When 
not given this information, protests and informal complaints 
were used to gain access to information as to why their 
proposal wasn't selected [18: p.25]. Finally, the commission 
thought that by passing legislation requiring debriefing, the 
Government would get better proposals and have more 
credibility with competing contractors. Although the 
commission made a good case for legislation concerning 
debriefing of unsuccessful offerors, Congress did not come 
through with any appropriate legislation. 
b. H.R. 3161 

In 1991 the House of Representatives proposed a 
resolution to amend laws pertaining to federal procurement. 
The resolution, H.R. 3161, included a portion on IENS 
debriefing of unsuccessful offerors. The portion of the bill 
was introduced over concern that insufficient debriefings may 
actually increase award protests. [10: p.35] Offerors unsure 


of the fairness of the award decision or those just wanting to 
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know the reasons why they did not win contract award could use 
a protest as a tool to find out what an adequate debriefing 
would have provided. The bill was specific in regard to what 
should be covered by statute. The proposed amendment 
contained five sections: 


(a) When a contract is awarded on a basis other than price 
alone, unsuccessful offerors, upon their written request, 
shall be debriefed and furnished the basis for the 
selection decision and contract award. 

(b) In any procurement for a contract for an amount 
greater than $5,000,000, a debriefing under subsection (a) 
shall provide to the offeror requesting the debriefing, at 
least the following information if disclosure of such 
information would not compromise confidential business 
information of the awardee: 

(1) the basic proposed technical solution or 
configuration of the awardee; 

(2) the overall evaluated cost of the awardee and of 
the debriefed offeror; 

(3) the cost or price associated with the major 
components of the awardee’s proposal, including line item 
Mricing if practicable; 

(4) the overall ranking of the awardee and the 
debriefed offeror, and the combined technical and cost 
scores of the awardee and the debriefed offeror; 

(5) the technical point scores of the awardee and the 
debriefed offeror on the factors and  sub-factors 
identified in the solicitation; 

(6) a description of the rational for the award; and 

(7) reasonable responses to questions posed by the 
debriefed offeror as to whether source selection 
procedures set forth in the solicitation, the source 
selection plan, applicable regulations, and other 
applicable authorities were followed by the Government. 
(c) Each solicitation for a contract greater than 
$5,000,000 shall notify participating offerors that the 
categories of information described in subsection (b) may 
be disclosed by the Government in post-award briefings. 
(d) The failure of any agency to provide a debriefing 
Satisfying the requirements of subsection (b) shall be a 
matter subject to protest to any forum having jurisdiction 
Over protests relating to the procurement, and the protest 
forum, in its discretion, may order appropriate relief in 
any such protest. 

(e) In any case in which a debriefing is conducted under 
this section and the procurement subsequently is resumed 
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as a result of a successful protest against award or other 

reason, the contracting officer shall provide to each 

competing contractor all information subject to disclosure 

under this section that is necessary to ensure equity and 

fair competition in the resumed procurement." (10: p.15] 

This bill proposed not only that debriefings be 
conducted by federal agencies but provided a detailed 
framework for what should be included in those debriefings. 
This framework went far beyond what the current regulations 
require for debriefings. Debriefings based on this outline 
would no doubt reduce the lingering questions left in 
unsuccessful offerors minds as to why they didn’t win a 
contract award. This outline was never put to use however as 
the bill did not pass into law. 
C. aS 58 
The Senate introduced a bill in 1991 that was 

Similar to the House’s H.R. 3161. It was S. 1958. The bill 
had four main goals, one of which was the establishment of a 
meaningful debriefing for unsuccessful offerors [19: p.3]. 
The Senate bill added a requirement that H.R. 3161 did not 
contain. A request for a debriefing covered under the Senate 
bill must be made within ten days after contract award and the 
debriefing must take place within ten days of the request 
[1966s ene. The Senate cited the need for legislation on 
debriefings as a result of debriefings being inconsistently 


given and even when given, many times failing to meet the 


objective of conducting them [19: p.9]. Additionally, the 
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Senate conceded that by requiring meaningful debriefings, the 
number of protests filed as a result of no debriefings or poor 
debriefings would drop [19: p.10]. Thus, the cost and time 
lost due to a protest could be avoided by a proper debriefing. 
Industry officials stated at the Senate hearings that federal 
agencies were not providing useful information regarding the 
reasons for award to the winner and non-award to the 
unsuccessful offerors. They further stated that many times 
protests were used to obtain the information they could not 
get at the debriefing [19: p.9]. The Government Accounting 
Office confirmed all of this [19: p.9]. As was the case with 
H.R. 3161 though, S. 1958 was not passed and no legislation on 
debriefing unsuccessful offerors resulted from either bill. 
d. Section 800 Panel Report 

In the National Defense Authorization Act of FY 
1991 Congress directed DOD to establish an advisory panel of 
Government and industry experts to review all laws affecting 
DOD procurement. The panel was to submit a report through the 
Secretary of Defense in January 1993 that made recommendations 
to streamline defense acquisition. The report submitted to 
Congress is commonly referred to as the Section 800 Panel 
Report. One of the recommendations that the panel submitted 
was an amendment for section 2305(b) (4) (B) of Title 10 of the 
United States Code requiring regulations that address 


debriefing unsuccessful offerors [25: p.1-4]. The panel’s 


15 


recommendation was based on the American Bar Association's 
1989 Report on Bid Protests which found that prompt and 
meaningful debriefings help stem the tide of protests. The 
Bar Association report contained the results of a survey taken 
of protestors and their lawyers. The results showed that a 
number of protests could be avoided if timely and meaningful 
debriefings were conducted. [25: p.I-232] These debriefings 
would show that the procurement was done properly and would 
provide information why the unsuccessful offeror was not 
selected for ecemtract award" 

The Section 800 Panel thought that any statute 
concerning debriefing unsuccessful offerors should cover 
general policy objectives while the regulations should cover 
the details. The panel recommended that the statute require 
three things of the regulations. First, the regulations 
should set up criteria for determining when a debriefing is 
required. The panel recognized that debriefings are not 
essential for such actions as small purchases, and procurement 
actions where there are no Significant evaluation areas other 
than cost. Next the panel believed that regulations must be 
required to ensure that debriefings be held within 15 calendar 
days after award, whenever possible. The panel believed that 
the more timely a debrief is held the less likely it was that 
a protest would be filed. Lastly, the panel believed the 
regulations must require the debriefing to include the 


Strengths and weaknesses of the unsuccessful offeror’s 
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proposal. By providing Strengths and weaknesses the panel 
thought an unsuccessful offeror would better understand why he 
was not the winner and would thus be less inclined to file a 


protest. [25: p.I-69] The outcome of the Section 800 Report 


and it's recommendations on debriefing are yet to be 
determined. The report was presented to Congress in October 
1993 and is still being discussed. 
e. Federal Acquisition Regulation 

The Federal Acquisition Regulation (FAR) is a set 
of regulations that cover the acquisition of all goods and 
services for all federal agencies. The FAR covers debriefing 
of unsuccessful offerors in 15.1003 which provides that: 


(a) When a contract is awarded on the basis of other than 
price alone, unsuccessful offerors, upon their written 
request, shall be debriefed as soon as possible and 
furnished the basis for the selection decision and 
Fomeraect award: 
(b) Debriefing information shall include the Government’s 
evaluation of the significant weak or deficient factors in 
the proposal; however, point-by-point comparisons with 
other offerors’ proposals shall not be made. Debriefing 
shall not reveal the relative merits or technical standing 
of competitors or the evaluation scoring. Moreover, 
debriefing shall not reveal any information that is not 
releasable under the Freedom of Information Act; for 
example- 

(1) Trade secrets; 

(2) Privileged or confidential manufacturing processes 
and techniques; and 

(3) Commercial and financial information that is 
privileged or confidential, including cost breakdowns, 
profit, indirect cost rates, and similar information. 
(c) The contracting officer shall include a summary of the 
S@eertefing in the contract file. [7: p.15-41] 


It is important to note what information the FAR 


allows the debriefing agency to release. Companies that 
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request debriefings are generally looking for as much 
information as possible to explain why their proposals were 
not selected for contract award. The FAR does allow general 
comparisons to be made between proposals. However, the 
debriefing will not cover the relative merits or technical 
Standing of competitors or the evaluation scoring. [7: p.15- 
41] A definition of relative merits, technical standing and 
evaluation standing is needed as these terms can be 
interpreted differently by different people. Dr. David Lamm 
of the Naval Postgraduate School describes what these three 
terms mean. Relative merits refer to the approaches taken by 
an offeror in his attempt to satisfy the evaluation criteria 
of the RFP. This includes the evaluated quality of those 
approaches. Technical standing is the ranking of the 
technical area or areas with respect to other offerors. It 
could be in the form of scores or numerical ranking. Lastly, 
evaluation scoring is the numerical score given to each area 
and total numerical score of all areas. [11] Thus, the FAR 
emphasizes the discussion be on the debriefed offeror’s 
proposal and discourages any discussion on other competitor’s 
proposals. 
f. Army FAR Supplement 

The Army FAR Supplement (AFARS) provides 

additional guidance to AMC for debriefing unsuccessful 


offerors. It consists of three sections which provide that: 
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a. The contracting officer will release to unsuccessful 
offerors the required information as prescribed in FAR 
M5. 1001. 

b. When debriefings are requested they shall be in 
accordance with FAR 15.1003. 

(1) Debriefings will be with only one offeror at a time 
and will not be conducted until after contract award. 

(2) The debriefing must be confined to a discussion of 
the offeror's proposal and 1ts advantages and 
disadvantages in relation to the requirements of the RFP. 
Comparisons should not be made relative to the proposal of 
other offerors. 

(3) No information will be disclosed to an offeror as 
to the weights or ratings assigned. 

C. Normally, the debriefings should be conducted by or 
under the direction of the contracting officer. It should 
be noted that some contractors may wish to discuss the 
outcome at higher levels of authority. Although this 
should not be encouraged, neither should barriers be 
placed in their way of getting an explanation from 
responsible officials. [1: p.10-3] 


This places further restrictions on the debriefing 
process. Of particular note is that the AFARS prevents 
general comparisons with other offerors’ proposals even though 
such comparisons are allowed by the FAR. The AFARS also 
states that debriefings should be limited to the offeror's 
proposal. This prevents any discussion of the winning 
offeror’s proposal. However, the agency that wrote this 
guidance stated that each debriefing should be treated on a 
case by case basis and that contracting commands have the 
latitude to deviate from this guidance. The reason this 
guidance was written was that there had been problems with 
unsuccessful offerors getting some information on the winner’s 
proposal evaluation and then trying to re-engineer the source 
Selection evaluation process. With the limited information 


that some unsuccessful offerors received they were sometimes 
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coming up with the conclusion that they couldn't have lost the 
procurement competition and then protested the procurement. 
The unsuccessful offeror will never understand the complete 
selection process because proprietary business data cannot be 
released. Therefore he will never have a complete picture of 
the source selection process. [20] Additionally, the AFARS 
stipulates that contracting officers debrief one offeror at 
a time and only after contract award. The AFARS highlights 
the contracting officer’s role as the official interface 
between his command and the defense contractor and it requires 
that the debriefing should be conducted or directed by him. 
g. AMC-P 715-3 Vol. 1 
AMC has it’s own publication which discusses 
source selection and it contains a section on debriefing. 
AMC-P 715-3, Vol. 1 reiterates the same guidance that the 
AFARS provides except it adds that: 
normally the Debriefings should be conducted by or 
under the direction of the [contracting officer], although 
the manner in which debriefings are to be handled is 
discretionary with the SSA. [2: p.51] 
This reinforces the role the SSA has in the source selection 
process to include the last communications with the 
unsuccessful offerors, that of the debriefing. The SSA can 
influence the debriefing in ways the contracting officer 
cannot. The SSA can provide resources for the debriefing that 


may not be available without his help. This is especially 
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true for SSEB members who can improve the debriefing by 


participating in it. 


C. WHY DEBRIEF UNSUCCESSFUL OFFERORS 

There are several reasons why the Army should debrief 
unsuccessful offerors. The first reason is because it is 
required. The FAR states that when a contract is awarded on 
a basis of other than price alone unsuccessful offerors will, 
upon written request, be debriefed as soon as possible and be 
provided the basis for the selection decision [7: p.15-41]. 
The FAR makes no mention of the size or complexity thresholds 
that need to be met before debriefings will be held. TE 
merely states that if one or more factors, other than price, 
is used to determine the source selection, then unsuccessful 
offerors are entitled to a debriefing when they ask for one in 
writing. Thus, all federal agencies are required to debrief 
all unsuccessful offerors that request a debriefing in 
writing. 

Another reason to debrief is to help improve the future 
competitiveness of the unsuccessful offeror. By providing the 
unsuccessful offerors with the rationale behind why they 
weren't selected for contract award they will be able to 
submit better proposals for future contracts [17: p.528]. 
Commenting on all evaluated areas and identifying strengths 
and weaknesses of the offeror's proposal will allow him to 


capitalize on his strong areas and improve on his weaknesses 
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for future proposals. This should improve his competitiveness 
for any subsequent solicitations he responds to. Without this 
explanation the unsuccessful offeror may make false 
assumptions about what improvements he needs to make and thus 
may continue to commit the same costly errors on future 
proposals. Improving his competitive position also helps the 
buyer as it receives a more competitive proposal in future 
procurements. 

Fairness is another reason why the unsuccessful offeror 
should be debriefed [3: p.1]. The contractor worked hard on 
getting his proposal done and spent precious company resources 
in the process. For large solicitations his best efforts to 
win the contract award may result in months of work and 
hundreds of thousands of dollars spent preparing a proposal. 
The interests of fairness dictate that the contractor get some 
feedback as to why he wasn’t selected for contract award. It 
is only fair that federal agencies respond to the contractors 
efforts and spend some time explaining how they evaluated the 
proposal. 

Next, debriefings can show unsuccessful offerors that the 
source selection decision was rational and conformed to the 
requirements and evaluation criteria set out in the RFP 
[17: p.528]. With no feedback the unsuccessful offeror would 
less likely know for sure if the source selection was based 
solely on the evaluation criteria in the RFP and if the 


evaluation board used the criteria in a logical way to 
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evaluate it’s proposal. By providing a debriefing the 
unsuccessful offeror can see that the evaluation committee did 
indeed use all the criteria identified in the RFP and 
evaluated each area in accordance with the standards set forth 
for each area. 

Lastly, by providing a debriefing contracting officials 
should reduce the possibility of protests being filed which 
could delay the procurement [3: p.1]. By placing their "cards 
on the table" unsuccessful offerors can see that the contract 
agency has nothing to hide and is acting in good faith. 
Additionally, they could get the information they are looking 
for to find out why they lost without having to file a 
protest. Thus, a protest would less likely be filed as a 
result of providing a debriefing. This is the reason the 
Section 800 Panel cited as it’s basis for requiring 


debriefings. 


D. WHERE DEBRIEFING FITS IN THE SOURCE SELECTION PROCESS 

The debriefing process fits in near the end of the source 
selection process but is not an isolated event. Rather, all 
steps in the process have some effect on the debriefing of 
unsuccessful offerors. 

The source selection process begins with the development 
of the source selection plan (SSP), which is normally 
comprised of two parts dealing with the source selection team 


membership and the evaluation criteria. The first part covers 
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the details of the organization, membership and 
responsibilities of the source selection party. It describes 
who is on the Source Selection Advisory Council (SSAC) and 
Source Selection Evaluation Board (SSEB), if it is used. The 
advisory council's role is to advise the SSA on the source 
selection process and actual source selection. The SSA is the 
person who is responsible for the entire source selection 
process. Two of his important duties are to ensure the source 
selection plan is properly written and executed and to select 
the winning proposal based on the information the SSEB and 
SSAC give him [5: p.10-B-2]. The SSAC may provide the SSA 
with a comparative analysis of the evaluation results. Care 
must be taken to select personnel for the advisory council who 
have the appropriate skills and experience to properly advise 
the SSA. The SSEB is responsible to evaluate all proposals in 
accordance with the evaluation criteria set in the SSP and 
RFP. The SSEB members must have the proper experience and 
expertise to fully understand the evaluation criteria and make 
competent evaluations of proposals in accordance with those 
criteria. Selection of the right people to fill these 
positions will result in the thorough evaluation of each 
proposal and the proper results given to the SSA. This in 
turn will result in the correct proposal receiving contract 


award and meaningful comments on the strengths and weaknesses 
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of all other proposals for debriefing of unsuccessful 
offerors. 

The second part of the source selection plan covers the 
evaluation criteria. The SSA and SSAC are responsible for the 
development and weighting of the evaluation criteria. The 
criteria are used by the SSEB to evaluate each proposal. All 
proposals are evaluated in the same manner to ensure a 
consistent process. The evaluation criteria must be selected 
and written to ensure the Government gets the product that 
best meets the solicitation and statement oof work. 
Establishing a method to evaluate each criteria is extremely 
important as it serves as the basis for the evaluation and 
justification of each rating. Evaluation criteria are to be 
measured quantitatively and qualitatively and must be written 
as such. There should be a standard against which each 
criteria can be judged. Additionally, the evaluation criteria 
must be properly weighted to reflect the relative importance 
of each criteria as determined by the SSA and “SSAC. 
Incomplete or poorly worded criteria or improperly weighted 
evaluation criteria can result in an inaccurate assessment of 
proposals. Problems with source selection and subsequent 
debriefings would then follow. Narrative comments extracted 
from the evaluation for use in a debriefing may then be 


incorrect, misleading or incomplete. This results in a 
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debriefing that is inaccurate or lacks any specifics. These 
narrative comments are crucial to a successful debriefing. 
They provide the concise information unsuccessful offerors 
need to understand what their strengths and weaknesses were in 
each area. Establishing comprehensive and accurate standards 
for evaluation criteria and weighting them properly will 
prevent any problems evaluating proposals and instead yield 
correct and meaningful evaluations and comments that can be 
used to make a correct source selection and improve the 
debriefing. 

Next, a competitive range is determined. This is normally 
determined by the contracting officer and approved by the SSA. 
Three areas are looked at when evaluating proposals to 
determine if they are in the competitive range. First, the 
proposal must be determined to be responsive. In the terms of 
negotiated procurement the proposal is considered to be 
responsive unless the offeror refuses to comply with any of 
the requirements of the RFP. The other two areas to be looked 
at are the technical merits and cost of the proposal. 
Proposals are to be deemed in the competitive range with 
regard to these two areas as long as there is a possibility 
that the proposal can win the contract award [9: p.4-9]. When 
a proposal falls out of the competitive range the offeror is 


notified. Debriefings resulting from being excluded from the 
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competitive range may not be as detailed as debriefings given 
to offerors who make it all the way through to best and final 
offers (BAFO's). A comprehensive evaluation may not have been 
needed to determine the proposal was outside the competitive 
range. Discussions are held with those offerors in the 
competitive range. Meaningful discussions should have a 
direct impact on the debriefing of unsuccessful offerors. 
Meaningful discussions will identify any deficient areas in a 
proposal and help clarify any questions the evaluation team 
has with respect to the proposal. This allows the contractor 
to make modifications to his proposal to increase it’s 
competitiveness. Additionally, productive discussions should 
head off any surprises in the debriefing regarding weak areas 
in the proposal as these weaknesses would have already been 
identified and then addressed by the offeror. 

BAFO's are received after discussions are concluded. At 
this time all offerors still in the competitive range may 
Submit final changes to their proposals and make a final 
offer. A final evaluation is done on all those still in the 
competition. BAFO’s may change the evaluation of those areas 
amended and thus influence the award and subsequent 
debriefing comments of those areas. 

Next, a source selection decision is made, contract award 


1s completed and unsuccessful offerors are notified. 
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Information given out during notification includes the number 
of RFP's sent out and proposals received, the contract price, 
and, in general terms, the reason why the offeror's proposal 
was not the winning proposal [7: p.15-41] This concludes the 
source selection team's interaction with the unsuccessful 
offeror, unless they take part in the debriefing or a protest 
is filed. 

For solicitations that do not require a formal source 
selection team the contracting officer plays a bigger role in 
the process. Evaluation criteria may be written by the 
contracting officer and a technical and cost analyst. The 
proposal evaluations may be done solely by the contracting 
officer or may involve a small team of evaluators, such as a 
technical evaluator and cost analyst. Even though the process 
is smaller and less complicated the impact that each part of 
the source selection process has on debriefing unsuccessful 


offerors is the same. 


E. ARMY MATERIAL COMMAND 

AMC is an Army major command headed by a four star 
general. It’s basic mission is to develop, buy and maintain 
material for the Army [24: p.2]. It is composed of depots, 
laboratories, arsenals, manufacturing and maintenance 


facilities, proving grounds, test ranges, and buying offices. 
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There are currently six commodity commands and four functional 
commands within AMC. These units are responsible for the 
research, development, testing, acquisition, and maintenance 
of the Army's material resources. 
1. Functional Commands 
The four functional commands include the Army Research 
Laboratory (ARL), the Depot System Command (DESCOM), the 
Simulation Training and Instrumentation Command (STRICOM), and 
the Test and Evaluation Command (TECOM). 
a. ARL 
The ARL is a command composed of research centers 
that generate new technologies and advanced concepts for all 
existing and future Army equipment. 
b. DESCOM 
DESCOM is responsible for the operation of the 
Army’s depots. The depots receive, store, issue, perform 
maintenance on and dispose of Army equipment. 
C. STRICOM 
STRICOM’s mission is to manage and direct the 
Army’s Simulation, training and test instrumentation needs. 
In addition it does the same for the new Distributed 


Interactive Simulation technology that the Army is using. 
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d.  TECOM 
TECOM is the command responsible for the planning 
and test and evaluation of the Army's equipment. This 
includes managing all testing facilities and equipment. 
2. Commodity Commands 

The six commodity commands include the Armament, 
Munitions, and Chemical Command (AMCCOM), the Aviation and 
Troop Command (ATCOM), the Communications-Electronics Command 
(CECOM), the Missile Command (MICOM), the Tank-Automotive 
Command (TACOM), and the Chemical and Biological Defense 
Command (CBDCOM). Each of these MSC’s provide research and 
development, engineering, procurement, and logistics support 
to their commodity areas. Additionally they provide matrix 
support to the Program Executive Officers of their respective 
commodity areas. These MSC’s do almost all of the acquisition 
actions within AMC. Together they spent over 95 percent of 
AMC's acquisition dollars in 1993 [15]. 

a. AMCCOM 

AMCCOM is responsible for armament, ammunition and 

chemical defense equipment. Armament includes towed and self 
propelled artillery, aircraft and tank armament, mortars, fire 
control systems, gun-type air defense weapons, and infantry 
weapons. Ammunition includes all ordnance for weapons systems 


from small arms infantry weapons to large artillery and tank 
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rounds. Other items include bombs, rocket and missile 
warheads, fuzes, mines, grenades, pyrotechnics, smoke, and 
incendiary devices. The AMCCOM organization includes the 
Research, Development and Engineering Center, three arsenals, 
15 active ammunition plants, and Acquisition Center. 
b. ATCOM 

ATCOM is responsible for aviation assets and troop 
support items. Aviation assets include Army aircraft, 
aircraft engines, spare parts, air traffic control hardware 
and aviation ground support equipment. Troop support items 
include all clothing and protective clothing items, footwear, 
food, food service equipment, water supply equipment, 
petroleum distribution equipment, all aerial delivery systems 
such as parachutes and slings, generators, Army watercraft, 
and rail and bridging equipment. Major activities in ATCOM 
include three research, development and engineering centers, 
a weapon system management center, an integrated material 
management center and an acquisition center. 

C. CBDCOM 

CBDCOM is responsible for the commodity management 
Of all chemical and biological related items. This includes 
protective suits, footwear and masks, detecting and monitoring 


equipment, and decontamination equipment and materials. Major 


ou 


elements of CBDCOM are a research, development and engineering 
center and acquisition center. 
d. CECOM 

CECOM is responsible for all Army communications 
and electronics equipment. Items include radios and radio 
related equipment, electronic warfare items, photographic 
equipment, radars, automated data processing and management 
information systems items, batteries, ground and airborne 
Surveillance items, night vision equipment, and satellite 
systems. Major activities of CECOM include the Research, 
Development and Engineering Center, the Command, Control, 
Communications, and Intelligence Logistics and Readiness 
Center and the Acquisition Center. 

e. MICOM 

MICOM is responsible for all equipment related to 
rocket and missile systems. This includes all components and 
subsystems related to air-to-ground missile systems, surface- 
to-surface missile systems, surface-to-air missile systems, 
ballistic missiles, and anti-armor missile systems. Major 
units in MICOM include a research, development and engineering 
center, an integrated management center and acquisition 


center.: 
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f. TACOM 

TACOM is responsible for commodity management of 
all combat and tactical vehicles. This includes tanks, all 
Carriers to include those for personnel, cargo, and missile, 
self propelled artillery vehicles, wheeled vehicles, special 
purpose equipment, trailers, construction equipment, and 
materials handling equipment. Major elements of TACOM are the 
Research, Development and Engineering Center, Integrated 
Material Management Center, Weapon Systems Management Center, 


and Acquisition Center. 


F. CONTRACTING OFFICERS IN THEIR ORGANIZATION 

The AMC MSC’s are each composed of several major centers. 
Centers common to the MSC's include the Research, Development, 
and Engineering Center, the Integrated Material Management or 
Logistics Center, and the Acquisition Center. The 
contracting officers that procure equipment and services are 


located in the Contract Operations Directorate of the 


Acquisition Center. The Acquisition Center is composed of 
several directorates. These are the Contract Operations, 
Acquisition Policy, Production Management, and Quality 


Management Directorates. The Contract Operations Directorate 
is split up into sections by commodity line and contracting 


officers are assigned to each section. 
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G. SUMMARY 

This chapter identified the laws and regulations that 
cover debriefing unsuccessful offerors, the reasons to 
debrief, where debriefing fits in the source selection 
process, what AMC is, and where contracting officers fit in 
their organization. An understanding of these topics will 
allow the reader to better understand how debriefing 


procedures can be improved. 
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III. METHODOLOGY 


A. GENERAL 

The methods used in the Thesis to collect information and 
data include a literature review, two survey questionnaires, 
and personal interviews. The literature search was done first 
to collect written material concerning debriefing unsuccessful 
offerors. This material and a personal interview with a 
Principle Assistant Responsible for Contracting (PARC) were 
used to develop the two Survey queStionnaires. Next, surveys 
were sent out to collect data concerning the debriefing 
process. One survey was developed for defense contractors. 
The other was written for AMC contracting officers. The two 
Surveys were very similar in nature and were geared to illicit 
responses to almost identical questions from the two sides 
participating in a debriefing. Personal interviews were 
conducted with defense contracting officials to verify the 
results of the survey and as a follow up to problem areas 
identified in the surveys that warranted further exploration. 
Finally, personal interviews were done with AMC contracting 
officers. These were done to verify survey results and to 
respond to potential solutions to problem areas that were 


identified in the preceding stages. 


J5 


B. LITERATURE SEARCH 

Literature research provided a background of information 
on debriefing. This information was then used as a framework 
for the rest of the thesis research. It was used to develop 
the surveys and personal interview questions. Literature 
research started with a search of the Naval Postgraduate 
School Library, Stanford Business School Library, the Defense 
Logistics Information Exchange, and Defense Technical 
Information Center. No major publications on debriefing were 
found as a result of this search. Instead, several works had 
small sections that pertained to debriefing. This provided 
only a small amount of information concerning debriefing. 
Next, a Search was done on Government publications and 
Congressional documents. This search resulted in two 
Government documents that discussed debriefing procedures. 
These are the Report of the Commission on Government 
Procurement, and Streamlining Acquisition Laws. Additionally, 
two Congressional reports were found. They are H.R. 3161, and 
S. 1958, which both represent Congressional attempts to pass 
legislation on the debriefing process. Lastly, the FAR and 
DOD regulations and instructions that pertained to debriefing 
unsuccessful offerors were collected. This provided the 
specific information regarding the current debriefing process 


Within AMC, 
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C. SURVEY QUESTIONNAIRES 
1. General 
Two survey questionnaires were designed and utilized 
to gather data on AMC’s debriefing procedures. One survey was 
constructed for defense contractors and the other for AMC 
contracting officials. 
2. Target Audience 
a. Defense Contractor Survey 
The target audience for the survey for defense 
contractors was a broad cross-section of defense contractors 


hat contract with AMC. This selection of defense contractors 


included large companies, small companies, small and 
disadvantaged companies, service companies, engineering 
companies, hardware companies, and software companies. 


Additionally, the selection of companies was done to ensure 
that all AMC MSC’s were represented in the survey target 
audience. 

The companies were selected froma list of defense 
contractors that appeared in the May/June 1993 edition of 
National Defense magazine. Several small companies, and small 
and disadvantaged companies were also selected from the MICOM 
source solicitation mailing list to ensure an adequate 
representation of these companies. Initially, a list of 


companies to send surveys to was being compiled by calling AMC 
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MSC's and asking for names of companies that were recent 
unsuccessful offerors. However this method was abandoned 
because of a concern that the MSC’s may have been providing 
names of "happy" unsuccessful offerors and not giving names of 
companies that were "unhappy" with the debriefing they 
received. This method of developing a survey mailing list 
would most likely have resulted in skewed survey results by 
understating any problems with the debriefing process. 
Therefore, the author used the National Defense magazine list 
of defense contractors, supplemented by the MICOM source 
solicitation mailing list to get a random list of defense 
contractors. This would result in more objective survey 
results. 

Surveys were sent to the director of contracts of 
each company. Instructions on the survey requested that 
personnel familiar with debriefings from officials within AMC 


and who received a debriefing from an AMC MSC within the last 


five years complete the survey. Surveys were sent to 76 
companies with a total of 204 surveys being mailed. Each 
company was sent at least two surveys. Ten companies were 


contacted by phone before surveys were sent to them to ensure 
that the surveys would be answered and returned. This was 
done to ensure a sample of surveys mailed would be returned 


for analysis. These companies indicated the number of surveys 
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that they wanted mailed to them. Of the 204 surveys mailed 
out, 41 were answered and returned, another 15 surveys were 
returned that were not answered with an explanation that the 
company either had not had a debriefing from an official 
within AMC within the last five years or they had not bid on 
any AMC contracts within the last five years. The goal for 
the number of completed surveys to be returned was 30. This 
is the number of samples needed to approximate the mean of the 
entire survey population according to statistics textbooks 
(26: p.298]. Even though statistical analysis was not the 
design of this survey, enough survey results were desired to 
ensure an approximation of results that the entire population 
of defense contractors that contract with AMC would have 
yielded. 
b. AMC Contracting Officer Survey 

The target audience for the AMC contracting 
officer survey was contracting officials within the AMC 
commodity commands that conduct debriefings for unsuccessful 
offerors. AMC MSC’s were contacted and Acquisition Center 
Chiefs and Directorate Chiefs provided the names of 
contracting officers within each command that conduct 
debriefings. A representative sample of contracting officials 
was provided for each of the five MSC’s studied in this 


thesis. Surveys were sent to the 42 contracting officials who 
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names were provided by their MSC. Responses were received 
from 32 officials. The goal was to get 30 responses. 
3. Survey Design 

The two surveys are almost identical in nature as all 
but three questions are the same. The surveys were designed 
to provide data on the respondent’s views of the AMC 
debriefing process. This two survey method is being used to 
compare answers between the two groups to see where 
Similarities and differences in opinion exist. They were 
tailored to yield sufficient information on the debriefing 
process while at the same time being short enough to ensure an 
adequate number of surveys were completed and returned. A 
copy of each survey appears in the appendix. The surveys have 
two major types of questions. The first type consists of 
multiple choice and specific fill in the blank questions and 
the second type consists of general open ended questions on 
the overall debriefing process. 

The first type of questions was designed to get 
responses to specific questions on the debriefing process. 
The defense contractor survey consists of 22 multiple choice 
questions and two specific fill in the blank questions. The 
Survey for AMC officials has 21 multiple choice questions and 
three specific fill in the blank questions. Possible answers 


on the multiple choice questions range from two to four with 
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the exception of one question which had six possible answers. 
These possible answers were chosen in order to categorize 
responses to aid in analyzing the survey data. The multiple 
choice questions each have a blank space for respondents to 
amplify their answers or make any comments in reference to the 
questions or their answers. The comments were to be used to 
aid in analyzing the answers and to see if any comments 
appeared repeatedly throughout the returned surveys. 

The second type of questions consisted of open ended 
questions. The defense contractor survey had five open ended 
questions while the AMC survey had four. The objective of 
using these questions was to illicit responses about the 
overall debriefing process and allow the respondents to make 
any comments they want to about the debriefing process in an 
attempt to get responses not covered in the first type of 
questions. These questions focus on the strengths and 
weaknesses of the debriefing process and how the process can 
be improved. These comments were used in an attempt to 
Capitalize on the strengths of the process and find ways to 


Overcome the weaknesses identified. 


D. INTERVIEWS 
Interviews were conducted with both defense contractor 


representatives and AMC contracting officials. The intent of 
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the interviews was to follow up on recurrent answers or 
comments from the surveys and to identify other areas of the 
debriefing process that could be improved. Interviews of 
defense contractors were conducted within the broad cross- 
section of companies that contract with AMC. This includes 
large and small businesses, hardware producers, and a services 
company. Interviews were conducted first with the defense 
contractors. This was done so that information on areas that 
needed improvement and their possible solutions could be 
collected and then presented to AMC contracting officials. 
These officials could then comment on the identified areas and 
feasibility of the possible solutions. Additionally, they 
could address any areas they wanted to and present possible 
solutions to any weak areas of the debriefing process. 
Interviews were conducted with contracting officers from three 
of the five MSC’s that were surveyed. 

All interviews are recorded as anonymous entries in the 
thesis. This was done to ensure more complete and honest 
comments would be obtained by not identifying the source of 
the comments. This avoids the concern on the part of 
interview participants that their comments could be viewed 


negatively by any others in the AMC procurement system. 


42 


E. SUMMARY 

This chapter outlined the eens of research used in the 
thesis. They include a literature search, survey 
questionnaires, and interviews. It identified the purpose for 
each method. The target audience and survey design for the 
survey questionnaires are fully explained. Lastly, details of 
how the interviews were done was covered. Chapter IV will 
present and analyze the data obtained from the surveys and 


interviews. 
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IV. DATA PRESENTATION AND ANALYSIS 


A. GENERAL 

This chapter contains the data presentation and analysis. 
It contains two major sections. The first is data 
presentation and analysis for the survey questionnaires sent 
out and the second covers interview comments and analysis of 


these comments. 


B. SURVEY QUESTIONNAIRES 

The questions are discussed as they appear on each of the 
surveys. A discussion of the answers and an analysis of the 
answers follow. The questions from the two surveys that are 
Similar are presented and analyzed together. 

T: Question 1 The intent of this question is to 
determine what percent of unsuccessful offerors request a 
debriefing. 

a. Question to unsuccessful offerors 
When not awarded a contract I request a debriefing 
more than 2/3's of the time 


between 1/3 and 2/3's of the time 
less than 1/3 of the time 


naw 
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1. Discussion 

Forty two (42) contracting officials answered 
this question. Sixty (60) percent responded that they request 
a debriefing more than 2/3's of the time, 24% answered that 
they request a debriefing between 1/3 and 2/3's of the time 
and 16% reported that they wanted a debriefing less than 1/3 
of the time. Many comments were included with the multiple 
choice answers. Seventeen (17) percent of the respondents 
reported that they always request a debriefing, 10% commented 
that the value of the contract influences their decision to 
request a debriefing. The larger the contract the more apt 
they are to request a debriefing. Finally 5% of the 
respondents said that they request a debriefing only when they 
don’t understand why they didn’t win the contract award. 

b. Question to AMC 


How many unsuccessful offerors ask for a 
debriefing? 


a. less than 1/3 
b. between 1/3 and 2/3's 
e more than 2/3's 
1. Discussion 
Thirty two (32) respondents answered this 
question. Seventy five (75) percent reported that more than 
2/3's of unsuccessful offerors ask for a debriefing, 12% 


commented that between 1/3 and 2/3’s of unsuccessful offerors 


ask for a debriefing and 12% stated that less than 1/3 of 
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unsuccessful offerors ask for a debriefing. Several comments 
were added to the answers. Sixteen (16) percent of the AMC 
officials wrote that usually all unsuccessful offerors ask for 
a debriefing. Nineteen (19) percent reported that for large 
solicitations normally all unsuccessful offerors ask for a 
debriefing. 

c. Analysis 

The feedback from the two surveys is very similar. 
Both surveys report that a high percent of unsuccessful 
offerors request debriefings. Additionally, both surveys have 
comments that support the fact that as a solicitation 
increases in value the debriefing for an unsuccessful offeror 
becomes more important. The results of this question confirm 
that debriefings are an important source of information for 
unsuccessful offerors. 
2. Question 2 

The intent of this question is to determine if AMC 
officials give debriefings when they are requested. 

a. Question to unsuccessful offeror 


The Army gives a debriefing every time I request 
one. 


yes 
no 


oO py 
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1. Discussion 
Forty two (42) unsuccessful offerors answered 
this question. Eighty eight (88) percent reported that they 
always get a debriefing and 12% answered that they don't 
always get a debriefing when they request one. Four comments 
were received. One contractor that answered no stated that it 
depends on the commodity command and another reported that he 
received one about 95% of the time he asked for one. Two 
comments were received from contractors who answered yes. One 
stated that AMC officials may try to do the debriefing on the 
telephone and the other said that some debriefings he receives 
are nearly worthless. 
b. Question to AMC 
I give a debriefing every time one is requested. 


yes 
no 


oO w 


1. Discussion 

Thirty two (32) answers were received. All 32 
AMC contracting officials reported that they always gave a 
debriefing when one was requested. Three respondents 
commented that they feel a refusal to give a debriefing may 
lead to a protest. One official commented that this is 
required by the FAR. One stated that they know of no reason 
not to give a debriefing if one is requested, and another said 


that formal face to face debriefings are rare. 
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C. Analysis 
There is a difference in answers between the two 
participants here. Twelve (12) percent of the contractors 
stated that they didn't always receive a debriefing when they 
requested one and all the AMC contracting officials reported 
that they always gave a debriefing when one was requested. 
The true answer is probably somewhere in between. The AMC 
contracting officers may not want to admit to not giving a 
debriefing or may have misunderstood a request for one as 
something else. It is also possible that some requests never 
get to the contracting officer and the contractor didn't 
follow up on the request to ensure the contracting officer was 
aware that a debriefing was requested. Additionally, there 
may be some mistake on the contractors part as to when a 
debriefing is authorized and they may request debriefings when 
one is not allowed. They would then not get one but think 
they were entitled to one. 
3. Question 3 

The intent of this question is to determine who 
attends debriefings. 

a. Question to unsuccessful offeror 


The following personnel (by job title) from my 
company attend the debriefing : 
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1. Discussion 
Many answers were given for this question. 
All answers included more than one person. The most 
frequently named people were the director of contracting, the 
business development/program manager and the director of 
engineering. Results of this question are shown in Table 1. 


TABLE 1: DEFENSE CONTRACTOR PERSONNEL THAT ATTEND DEBRIEFINGS 


e p 





Job Title Times Mentioned 


NO 
ad 


Pm ector of Contracting 


Business Dev/Program Manager 


ul 


Director of Engineering 


WW 


Proposal Manager 

Marketing Manager 

Vice President 

President 

Vice President- Engineering 
Pricing Manager 


Director of Operations 


e |e je je |N 
H TH 





Department Manager 
Peoneract Administrator 


Legal Counsel 





Director of Technology 
RCE: DEVELOPED BY RESEARCHER 


Nn 
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b. Question to AMC 


The following personnel (by job title) from my 
organization attend the debriefing ; 
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1. Discussion 

Many people were mentioned in the answers 
received for this question. The most common responses were 
the contracting officer, listed by all respondents, the 
contract specialist, and SSEB chairman and factor chairs. 
Four of the responses listed the contracting officer as the 
sole person who attends from their organization. Results of 
this question are annotated in Table 2. 


TABLE 2: AMC PERSONNEL THAT ATTEND DEBRIEFINGS 
















| Job Title 


Contracting Orf ICer 
| Contract Specialist 


Times Mentioned 


me je 
a je IN 


a Ly 


| tega1 couneis 
Price Analyst 


SOURCE: DEVELOPED BY RESEARCHER 
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C. Analysis 
The results of this question show that the 
debriefing is not a one man show for either side. Rather many 


people attend debriefings and in many cases the debriefing is 
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seen as important enough that SSEB chairman and vice 
presidents of companies attend. The job titles of employees 
that attend for unsuccessful offerors indicates that the firms 
view the debriefing as an integral part of their effort to 
understand why they lost the contract award and identify what 
steps they can take to be more competitive in future 
solicitations. 
4. Question 4 

This question was designed to determine if AMC limits 
the number of people who may attend debriefings. 

a. Question to unsuccessful offeror 


The Army limits the number of employees that may 
attend the debriefing from my company. 


a. always 
b. sometimes 
E never 
1. Discussion 
Forty one (41) offerors answered this 
question. Ten (10) percent responded that the Army always 
limits the number of employees, 59% replied that they 
sometimes limit the number, and 30% stated that the Army never 
limits the number of employees that can attend debriefings. 
Three comments were received on this question. Two stated 
that employee numbers were only limited if the conference room 
wasn’t big enough to handle more people. One contractor 


stated that this was infrequently a problem. 
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b. Question to AMC 


I limit the number of employees that unsuccessful 
offerors may send to a debriefing. 


a. always 
b. sometimes 
c. never 
1. Discussion 
Thirty one (31) AMC respondents answered this 
question. Three (3) percent stated that they always limit the 
number of employees, 39% said they sometimes limit the number, 
and 58% stated that they never limit the number of employees 
that may attend a debriefing. There were several comments 
made to this question. Three Contracting officials wrote that 
sometimes the conference room isn’t large enough, especially 
on large procurements were many participants from both sides 
attend. Six people stated that the number of employees the 
contractor sends has never been a problem. One stated Chat 
the available space in the conference room dictates the number 
that may attend. One respondent wrote that limiting spaces 
insures that only those people necessary to understand why 
their company lost the contract award attend the debriefing. 
Two comments referred to lawyers attending. One person stated 
that he didn’t limit the number of people attending but 
requested a list of people by job title who would attend the 
debriefing. This is done to determine if a lawyer is going to 


attend the debriefing, which would indicate a hostile 
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contractor. Another person said that limiting the number of 
people attending depends on if you know whether or not the 
debriefing will be hostile as you don't want to brief lawyers 
who are looking for free discovery. 
C. Analysis 

Both the unsuccessful offerors and AMC officials 
report that there are times when the number of employees from 
the unsuccessful firm that may attend a debriefing are 
limited. The unsuccessful offerors state that it happens at 
a much higher rate than AMC officials claim it happens. Only 
30% of the contractors said they were never limited to the 
number of employees that could attend a debriefing while 58% 
of the AMC officials stated they never limit the number of 
employees. The true answer is probably somewhere in between. 
The discrepancy between these percentages may be partially 
explained by a misunderstanding on the part of contractor 
personnel of who limits the number of people attending. In 
some cases it may be their own company limiting the numbers in 
an attempt to keep travel costs down. Additionally, some AMC 
officials may not have provided an accurate answer in an 
attempt to downplay any effect from limiting the number of 


employees that may attend a debriefing. 
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54 Question 5 The intent of this question is to 
determine if AMC officials are limiting, by job title, the 
employees that may attend debriefings from unsuccessful firms. 

a. Question to unsuccessful offerors 


The Army limits the number of employees, by job 
position that may attend the debriefing from my company. 


a. always 
b. sometimes 
E never 
1. Discussion 

Thirty nine (39) answers were received for 
this question. Three (3) percent responded that AMC always 
limits the number, by job position, 16% stated that they are 
sometimes limited, and 82% said that they are never limited by 
job position. Two comments were included with the answers. 
One stated that they were sometimes limited to sending the 
senior staff, and the other said that they were not limited 
but AMC wants a list beforehand of who they are going to send. 


b. Question to AMC 


I limit employees, by job title, that unsuccessful 
offerors may send to a debriefing. 


a. always 
b. sometimes 
Cc. never 
1. Discussion 


Thirty (30) responses were received for this 


question. Seven (7) percent stated that they sometimes limit 
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employees, by job position, from attending a debriefing and 
93% said that they never limit the employees, by job title. 
Three respondents commented that the unsuccessful offeror may 
send any employees they want to. One stated that he never 
limits the employees, by job on: but does ask for a list 
of who will attend ahead of time. Another commented that he 
gets a list ahead of time and uses it as a going in position 
in order to prepare and organize a positive and professional 
debriefing. 
C. Analysis 
There is a disparity between the percentage of 
responses for the two sides that report a limit, by job 
position, of who may attend a debriefing from the unsuccessful 
firm. Only 7% of AMC personnel report ever limiting employees 
by job position while 18% of unsuccessful firms report that 
this has happened. Possible reasons for this difference could 
be a misunderstanding on the contractor’s part of who places 
this limit. In some cases it may be the company and not AMC. 
Additionally if this ever happened it would be the contractor 
who recalled it happening first and it may be of minor 
Significance to AMC personnel and thus forgotten. 
6. Question 6 This question was designed to determine 


how long after a request debriefings are held. 
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a. Question to unsuccessful offerors 


Debriefings are normally held within 
calendar days Ori Ae Mes e 


a. 0-10 
b. =IT-20 
e more than 20 
1. Discussion 
Answers were received from 42 contractors. 
21% responded that debriefings were held within 10 days. 
Forty eight (48) percent said between 11-20 days and the 
remaining 31% stated more than 20 days. Only one comment was 
reported. The contractor wrote that the time varied but was 


usually close to or in excess of 30 days. 


b. Question to AMC 


Debriefings are held within calendar days of 
EEQUCS iE 
a Ola 
b. 11-20 
kr mmore than 20 
1. Discussion 
Thirty one (31) AMC officials responded to 
this question. Eighty four (84) percent stated that 
debriefings were done within 10 days of request. The other 


16% Said it was done between 11-20 days. Many comments were 
written in response to this question. One respondent wrote 
that it is best to schedule debriefings as soon as possible so 


that SSEB members are available to assist. One said that he 
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is always prepared to debrief the day after notification of 
unsuccessful offerors. A delay gives the appearance that you 
had to stop and document why the offeror lost. Another 
respondent said that you should allow sufficient time to 
prepare adequately and still have the decision process fresh 
in mind. One contracting officer wrote that his debriefs are 
normally built into the acquisition cycle schedule and pre- 
scheduled for the first 3 days after contract award. Another 
AMC official said that he did it within 10 days of request, as 
early debriefings help offset possible protests. Three 
respondents stated that the time depends on the number of 
offerors and when both parties can get together. 
C. Analysis 

There iS a significant difference of opinion 
between the two parties as to how long after the request the 
debriefings are held. Eighty four (84) percent of AMC 
personnel said debriefings were held within 10 days of request 
while only 21% of the unsuccessful offerors reported that they 
were done within this same time frame. One reason for this 
difference could result from the requirement for the 
debriefing request to be written. If these written requests 
are mailed contractors may consider the clock to start on 
their request when they drop their request in the mail. AMC 


officials would most likely start the clock when they receive 
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the request. This reason alone would not solve the vast 
difference in opinion between the two parties on this issue. 
When a debriefing is done can have an effect on if a protest 
is filed or not. This in turn can have an effect on any 
subsequent debriefing given. Contractors have 10 days after 
contract award is announced to file a protest which will stop 
work on the contract. The survey shows if a debriefing is not 
given within these 10 days a contractor may file a protest 
because he may suspect some type of impropriety in the source 
selection decision process. A debriefing done within the 10 
day window may show the contractor there was no impropriety in 
the decision process and no protest iS necessary. 
Additionally, any debriefings given to unsuccessful offerors 
that have already protested the contract award may not be very 
informative because the contracting officer is on the 
defensive and is not looking to give the unsuccessful offeror 
information that could be used in the protest. 

Ja Question 7 The intent of this question is to 
determine when the unsuccessful offerors and AMC officials 
think the debriefing should be done. 

a. Question to Unsuccessful offeror 


Debriefings should be conducted within 
calendar days after contract award. 
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1. Discussion 
Thirty eight (38) contractors answered this 
question. Answers received ranged from 5 to 30 days. Fifty 


(50) percent of those that responded said that the debriefing 


should be done within 10 days. The average answer to this 
question is 14.4 days. Three different comments were 
included with the answers. Five contractors responded that 


the debriefing must be done within the limits of the time to 
protest. Two people stated that the sooner the better and 
that sooner may help head off unnecessary protests. Lastly, 
one official stated that he recognized the difficulty of 
getting all the AMC people together but earlier feedback is 
better. 

b. Question to AMC 


Debriefings should be conducted within 
calendar days after contract award. 


1. Discussion 

Twenty eight (28) AMC officials answered this 
question. The answers ranged from 4 to 30 days with the 
average being 14.5 days. Fifty three (53) percent said that 
debriefings should be done within 10 calendar days after 
contract award. Several respondents provided comments with 
their answer. Two stated that the FAR says to conduct the 
debriefings as soon as possible. One stated that the SSEB 


goes to the four winds after award so it is best to do it as 


59 


soon as possible. One official commented that the 10 day 
protest cut off is a factor, however SSEB support is the final 
determining factor. Another comment was that many protests 
are filed after a debriefing so the debriefing should be done 
quickly. Two officials said to let the offeror decide when 
the debriefing should be done and honor their request. 
c. Analysis 
The responses from both the contractors and AMC 
officials indicate that debriefings should be done in a timely 
manner. The average response for both groups show that 
debriefings should be done within 14 days of contract award. 
8. Question 8 The intent of this question is to 
determine who normally conducts the debriefing. 
a. Question to unsuccessful offeror 


normally debriefs me. 


a. the contracting officer 
b. the source selection evaluation board chairman 
c. a combination of both a £ b 
d. other 
1. Discussion 
Forty two (42) defense contractors answered 
this question. Twenty one (21) percent responded that the 


contracting officer debriefs them, 74% said that a combination 
of the contracting officer and SSEB chairman debrief them and 
5% stated that someone else debriefs them. This other person 


was the program manager. Several comments were received that 
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added names of others who participate. Lawyers were mentioned 
3 times, technical evaluators were listed on 4 surveys, cost 
analysts were mentioned 2 times and a contract specialist was 
listed once. One additional comment was that when the 
contract officer is the only person to debrief the debriefing 
tends to be dry and not very informative. 

b. Question to AMC 


Debriefings are conducted by (give job title) 


1. Discussion 
Five different answers were received for this 
question. Thirty eight (38) percent of the respondents 
indicated that the contracting officer conducts’ the 
debriefing. Nineteen (19) percent said the contracting 
officer and the contract specialist conduct the debriefings, 
and 19% stated the contracting officer and technical experts 
dommit. Twenty five (25) percent of the people said the 
contracting officer and the SSEB conduct debriefings. 
c. Analysis 
Both the contractors and AMC officials answers to 
this question reveal that a contracting officer participates 
in almost all debriefings. They are generally joined by such 
personnel as the SSEB chairman, SSEB members, technical 


representatives, and contract specialist. It is clear though 
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that the contracting officer is the key player for AMC in the 
debriefing process. This coincides with what the AFARS 
recommends. 

9. Question 9 This question was designed to determine 
where debriefings are held. 

a. Question to unsuccessful offerors 

Debriefings are normally held at 
at my location 
at the Army’s location 


other location 
by telephone 


aa oo 


1. Discussion 

Forty two (42) contractors answered this 
question with 2 contractors giving 2 answers. Eighty six (86) 
percent of the responses indicated that the debriefings are 
held at the Army’s location. Fourteen (14) percent of the 
answers reported that they are done by telephone. Two 
contractors listed both of these answers. Two different 
comments accompanied these answers. Three contractors who 
answered that debriefings are normally held at the Army’s 
location stated that sometimes telephone debriefings are done. 
One of these officials stated this is usually a conference 
call. Another of these officials said that the telephone 
debriefings are frequently done when the AMC command is out of 


town. 
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b. Question to AMC 
Debriefings are normally held 


at my location 

at the unsuccessful offeror's location 
other location 

by telephone 


aQna dy 


1. Discussion 
Thirty four (34) answers were received. 
Eighty eight (88) percent of the answers indicated that 
debriefings are normally held at the AMC location while 12% 
reported they are done by telephone. Two contracting officers 
stated both of these answers. Eight AMC personnel commented 
that debriefings are sometimes done by phone. Four of these 
people added that this was especially so for small 
procurements. Two officials stated that debriefings should 
never be done at the unsuccessful offeror’s location. 
C. Analysis 
The answers to this question are almost identical 
for each survey. The results indicate that most debriefings 
are done in person at the AMC location. A small percent of 
the debriefings are done by phone, especially when the 
procurement is small or when the AMC location is out of town. 
Not one respondent reported that any debriefing was ever done 
at the offeror’s location. All of this indicates that there 
1S an unwritten practice to not conduct debriefings at the 


offeror’s location. tt senewoOLrerone wants the service in 
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person he must travel to get it. The reason contractors will 
request debriefings by phone for smaller procurements is that 
they are not willing to pay for travel costs. The costs would 
outweigh the benefits in these cases. Instead they will get 
as much information as they can by telephone. 
10. Question 10 

The intent of this question is to determine if 
questions are allowed and answered in debriefings. 

a. Question to unsuccessful offeror 
The Army 
doesn’t allow questions 


allows and answers all questions 
allows and answers some questions 


aw 


1. Discussion 

Forty two (42) answers were received for this 
question. Seven (7) percent of the respondents said that the 
Army doesn't allow questions. Two (2) percent stated that 
questions are allowed and they are all answered. Ninety one 
(91) percent indicated that the Army allows and answers some 
questions. Four comments reported that the answers were not 
detailed enough. Two contractors commented that they were 
limited to asking questions about their proposal only. One 
offeror stated that allowing and answering questions really 


depends on who the contracting officer is. 
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b. Question to AMC 
I 
don't allow questions 


allow and answer all questions 
allow and answer some questions 


nu 


1. Discussion 
All 32 contracting officers answered this 
question. Thirty eight (38) percent stated that they allow 
and answer all questions and 62% said that they allow and 
answer some questions. Several comments were received. Eight 
respondents said that they don’t allow questions about another 
offeror’s proposal. Three stated that they answer questions 
to the extent the regulations allow. Two officials said that 
they may answer some questions in writing after the 
debriefing. One contracting officer stated that he can’t 
answer some questions and some aren’t appropriate. Another 
Said that all questions and answers must be in writing and 
that the openness of the debrief depends on the likelihood of 
a protest. Lastly, one official wrote that he uses a caucus 
between questions and answers. 
C. Analysis 
There is a large difference between the two 
surveys with the answer that the Army allows and answers all 
questions. Thirty eight (38) percent of AMC personnel gave 


this answer versus only 2% of the contractors. Some 
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interpretations of the possible survey answers may have been 
done to cause this. It doesn’t seem likely that AMC officials 
would answer all questions asked by contractors, especially in 
the contractor’s quest to get information on the winner’s 
proposal. Consequently a more accurate answer for AMC is 
probably much lower. Several contractors reported that they 
are not allowed to ask questions during a debrief. This 
doesn’t allow the contractor to get the information he needs 
to understand the evaluation of his proposal. 
11. Question 11 
This question was designed to determine why some 
questions asked by unsuccessful offerors are not answered. 
a. Question to unsuccessful offeror 


If the Army doesn't answer some of my questions it 
is because 


a. they can't answer them due to regulations or 
confidential/proprietary business information 
they won't answer them due to their concern for a 


protest 
C. other 
d. not applicable, they answer all my questions 
1. Discussion 
A total of 56 answers were recorded for this 
question. Although the question was designed to get only one 


answer from each respondent, 15 of the 41 officials who 
answered this question gave two responses. These 15 answered 


that the Army doesn’t answer some questions due to both 
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regulations and confidential/proprietary business information, 
and their concern for a protest. Fifty nine (59) percent of 
the answers indicated that regulations or 
confidential/proprietary business information is the reason 
the Army doesn't answer all questions. Thirty six (36) 
percent of the answers pointed to a concern for protests as 
the reason. Five (5) percent of the respondents listed other 
reasons which were that the question is not relevant or is 
source selection sensitive or classified, they don’t know the 
answer to the questions they don’t answer, and a hidden agenda 
prevents them from answering some questions. One comment was 
recorded. It stated that AMC personnel tend to error on the 
conservative side when determining when to answer a question. 
b. Question to AMC 


If I don’t answer some of the unsuccessful 
offerors questions it is because 


a. I Can't answer them due to regulations or 
confidential/proprietary business information 

b. I won't answer them due to a possible protest 

c. other 

d. not applicable, I answer all questions 

1. Discussion 
Thirty five answers were received for this 

question. Three AMC officials gave two answers. Eighty three 
(83) percent of the answers indicated that when questions 


aren't answered ie is due to regulations, or 


confidential/proprietary business information. Nine (9) 
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percent of the answers stated some questions aren't answered 
for other reasons. These reasons are when the offeror is 
seeking opinion and not facts, and if information is not 
available and then an attempt will be made to provide it 
later. Two surveys gave this latter reason as their answer. 
Contracting officers submitted several comments to this 
question. Two stated that they don’t worry about protests and 
answer aS many questions as possible. One of these continued 
that the Freedom of Information Act allows most information to 
be disclosed so you might as well give it to them up front. 
The other added that the Government Accounting Office will let 
a protestor see everything during discovery so he doesn’t hide 
anything. Another official stated that when there is the 
potential for a protest it makes his answers rather anemic. 
One official stated that he doesn’t answer questions that 
compare proposals, disclose relative rankings or discuss 
weights. 
c. Analysis 

Both sides agree that many questions are not 
answered due to regulations or confidential/proprietary 
business information. However a higher percentage of 
contracting officers than unsuccessful offerors stated this as 
the reason some questions aren’t answered. This is most 


likely a result of how the FAR and AFARS are interpreted. It 
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is the contracting officer’s responsibility to operate within 
the regulations, so they are much more likely to take a more 
conservative view of what these regulations mean. The two 
survey groups disagree on the issue of not answering questions 
due to a fear of being protested. Not one contracting officer 
answered that fear of a protest prevents them from answering 
a question while 46% of the contractors stated that this is 
the reason some questions aren't answered. This leads to the 
conclusion that contracting officers feel they are answering 
all they can legally answer while many contractors believe 
questions that could be answered are not being answered. 

12. Question 12 This question was designed to determine 
if contracting officers pass along all the information that 
unsuccessful offerors are entitled to according to the 
regulations. 

a. Question to unsuccessful offerors 
Debriefings normally consist of 
a. limited information, because 
D. as much information as the Army can give in 
accordance with the regulations and confidential/proprietary 
business information 
1. Discussion 
All 42 respondents answered this question. 
48% stated that the debriefings consisted of limited 
information. Fifty five (55) percent of these people said the 


reason was due to the fear of a protest. Two said it was due 
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to proprietary data rights and three others said it was due to 
restrictive regulations. Three contractors stated that it was 
because of an attempt to cover up improper procurement 
practices. The last reason given was that the contracting 
officers don’t want to compare the offeror with the winner. 
The other 52% of the respondents answered that the Army gives 
out as much information as it can under the current 
regulations and confidential/proprietary business information 
Stipulations. Two comments to the question were given. One 
was that certain contracting officers give very detailed 
debriefings while others give generic debriefings. The other 
stated that it depends on what MSC you're dealing with as some 
provide more information than others. 

b. Question to AMC 

Debriefings normally consist of 

a. limited information, because 

b. as much information as I can give in accordance 
with the regulations and confidential/proprietary business 
information 

1. Discussion 
Thirty one contracting officers answered this 

question and all stated that they give out as much information 
as they can in accordance with the regulations and 
confidential/proprietary business data. Many comments were 


provided with the answers. Five people stated that providing 


all the information you can helps prevent protests and other 
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problems such as congressional inquiries. One contracting 
officer stated that he only gives out the minimum information 
necessary. Another said that he gives only a summary 
debriefing and provides additional information if asked. One 
person stated that he thinks AMC is extremely conservative in 
what information it gives out. One comment indicated that 
contractors want information that compares their proposal to 
the winner’s. Lastly, one official said his command now gives 
out the same information to the unsuccessful offeror on his 
proposal that is given to the SSA. 
c. Analysis 

There is a large difference between what the AMC 
officials and the unsuccessful offerors believe as to the 
extent of the information given out in debriefings. All the 
contracting officers said that they give out what they can, 
(although some of their comments refute this) while almost one 
half of the contractors said that the debriefings consist of 
Gui limited information. While contracting officers 
apparently feel that they give out all the information they 
can there is some evidence that certain commands and 
contracting officers are willing to provide more information 
Co unsuccessful offerors than others. Factors such as fear of 
protests, conservative interpretations of the regulations, 


time constraints, and heavy work loads probably prevent some 


MEL 


commands and contracting officers from providing all the data 
they could provide under optimum conditions. 
ds Question 13 The intent of this question is to 
determine if the threat of a protest has any effect on the 
amount of information given out in a debriefing. 
a. Question to unsuccessful offerors 


The Army would provide more information if the 
possible threat of a protest did not exist. 


always 
sometimes 
never 
don't know 


e an OM 


. Discussion 

Forty two (42) answers were received for this 
question. Twelve (12) percent of the offerors believe that 
the threat of a protest always effects how much information is 
given out ina debriefing. Thirty eight (38) percent believe 
it sometimes has an effect and 2% believe it never has an 
effect. Forty three (43) percent don’t know if the Army would 
provide more information if the threat of a protest didn’t 
exist. Two respondents commented that if this wasn’t the case 
then the debriefings would be more comprehensive. 

b. Question to AMC 


I would provide more information if the possible 
threat of a protest did not exist. 


always 


a. 
b. sometimes 
c never 
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1. Discussion 

Twenty nine (29) CG@mtracting officers 
responded to this question. Seven (7) percent said that the 
threat of a protest always influenced how much information 
they provide in a debriefing. Twenty eight (28) percent 
stated that a protest sometimes influences them and 65% said 
that it never influences the amount of information they 

provide. 

C. Analysis 
Both contracting officers and unsuccessful 
offerors say that the threat of a protest influences the 
amount of information that is given out in a debriefing. 
Fully one third of the contracting officers say that it 
effects them at least sometimes. Therefore a conclusion can 
be made that the threat of a protest influences the quality of 
many debriefings. Contractors that are hostile or threaten a 
protest before the debriefing is conducted are thus less 
likely to get the same amount of information they would get if 
the contracting officer felt confident no protest would be 
filed. 

14. Question 14 This question was designed to determine 
if debriefings are conducted in accordance with the major 


elements of the RFP. 
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a. Question to unsuccessful offerors 


The Army debriefs in accordance with the major 
elements of the RFP. 


a. always 

b. sometimes 

C never 

1. Discussion 
All 42 contractors answered this question. 

Fifty (50) percent stated that the Army always debriefs in 
accordance with the RFP. Forty four (44) percent said that 
they sometimes do and 7% indicated that the Army never 
debriefs in accordance with the RFP. Two comments were 
submitted with the answers. One contractor commented that 
whether or not the debriefing is done in accordance with the 
major elements of the RFP depends on the command. Another 
Stated that only summary data is provided for the major 
elements. 


b. Question to AMC 


I debrief in accordance with the major elements of 
the RFP. 


a. always 
b. sometimes 
E never 
1. Discussion 
Thirty (30) AMC officials answered this 
question. Ninety three (93) percent stated that they always 


debrief in accordance with the RFP. Seven (7) percent said 
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that they sometimes do. One official commented that he 
usually limits the debriefing to only the weak or deficient 
factors in the proposal. Another stated that he gives 
narrative remarks for all factors and sub-factors. 
c. Analysis 

There is a large difference between the answers 
given by the unsuccessful offerors and the AMC contracting 
officers. Only 50% of the unsuccessful offerors believe that 
contracting officers always debrief in accordance with the RFP 
whereas 93% of the contracting officers believe they do. This 
difference could be accounted for by the amount of information 
Given out for each major element. If only a small amount of 
information is given out unsuccessful offerors may have 
interpreted this as not having been debriefed in accordance 
with the major elements while contracting officers may have 
considered this being debriefed in accordance with the major 
elements. 

15. Question 15 This question was designed to determine 
if debriefings clearly point out weaknesses in the 
unsuccessful offeror’s proposal. 

a. Question to unsuccessful offerors 


Debriefings clearly identify weaknesses in my 
proposal. 


always 


a. 
b. sometimes 
c never 
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1. Discussion 
Forty two (42) responses were given for this 
question. Twenty four (24) percent reported that the 
debriefings always clearly identify weaknesses in the 
offeror’s proposal. Sixty seven (67) percent said that they 
sometimes did and 9% indicated that they never clearly 
identified weaknesses in their proposal. Two comments were 
provided. They all indicated that improvements need to be 
made here. Seven unsuccessful offerors stated that there was 
not enough information given out to determine exactly what the 
weaknesses were in the proposal. One respondent commented 
that it seems like just enough trivial weaknesses are 

identified to justify a low cost award. 
b. Question to AMC 


Debriefings clearly identify weaknesses in the 
unsuccessful offeror’s proposal. 


a. always 
b. sometimes 
e never 
1. Discussion 
Thirty two (32) contracting officers answered 
this question. Eighty one (81) percent stated that they 
always clearly identify weaknesses in unsuccessful offeror’s 
proposals. Nineteen (19) percent responded that they 


sometimes clearly identify weaknesses. Several comments were 


added to the answers. Two contracting officers commented that 


76 


weaknesses or deficiencies not discussed prior to the 
debriefing should not be addressed during the debriefing. 
Another stated the same thing, but he added that he had an 
experience with a protest to GAO where GAO determined that any 
weaknesses that negatively affect the offeror’s proposal must 
be discussed during negotiations or GAO may determine 
meaningful discussions were not held. Another official 
commented that weaknesses are not always available on each 
element due to a lack of clear documentation in the technical 
evaluations. Two contracting officers commented that major 
deficiencies are always addressed but weaknesses are only 
addressed if asked or they had a significant impact on the 
source selection. The last comment was that weaknesses will 
be covered if they exist but it is extremely difficult to 
debrief a good proposal with no technical weaknesses. 
c. Analysis 

Several unsuccessful offerors and contracting 
officers stated that debriefings clearly identify weaknesses 
in many cases but not all cases. This means that not all the 
information that could be given to an unsuccessful offeror is 
always being given. Providing the offeror with his weaknesses 
allows the contractor to see how his proposal stood up to the 
evaluation criteria and more specifically where his proposal 


did not measure up to the standards expected of it in 
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accordance with the evaluation criteria. Clearly identifying 
the offeror's weaknesses in the debriefing benefits the 
contractor but it is not being done in all debriefings. 

16. Question 16. The intent of this question is to 
determine it debriefings cover the three major elements of 
most proposals. 

a. Question to unsuccessful offerors 


The Army debriefs me on the merits of my 
proposal (circle all that apply). 


a. technical 

b. management 

E cost 

1. Discussion 
This question allowed the respondents to 

circle all answers that apply to debriefings they have 
attended. A total of 113 answers were received with all 42 
offerors selecting at least two answers. Ninety five (95) 
percent of the offerors reported that the Army debriefs them 
on the technical merits of their proposal. Eighty six (86) 
percent stated that they are debriefed on the management 
merits and 79% said they are debriefed on the cost merits of 
their proposal. 


b. Question to AMC 


I debrief unsuccessful offerors on the 
merits of their proposal (circle all that apply). 


a. technical 
b. management 
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Es Ost 
1. Discussion 

The question was set up to allow the 

contracting officers to circle all answers that apply to 
debriefings they give. All 32 contracting officers answered 
the question and a total of 80 answered were recorded. Eighty 
seven (87) percent replied that they debrief the technical 
merits of a proposal. Eighty four (84) percent stated that 
they debrief the management merits and 78% said that they 
debrief the cost merits of a proposal. Several comments were 
included with the answers. Three people wrote that they cover 
all criteria advertised in the RFP. One official said that he 
also debriefs logistics, production, and integrated support. 
Another contracting officer stated that he covers whatever 
issues the unsuccessful offeror wants to discuss. Finally, 
one respondent stated that he doesn’t debrief cost as the 
unsuccessful offeror knows his price and the winner’s price. 

c. Analysis 

The large majority of both the unsuccessful 

offerors and contracting officers responded that debriefings 
cover the technical, management and cost merits of the 
proposal. It would seem though that these answers should be 
approaching 100% as almost all procurements that warrant a 


debriefing contain something in these categories to evaluate. 
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This is certainly true of price as it is a required 
consideration in all procurements. The procurement command 
should do some type of cost realism evaluation on each 
proposal submitted. To not debrief price indicates that the 
contracting officer is not passing along to the offeror the 
evaluation of the price portion of the proposal. By mice 
debriefing all three of these areas contracting officers may 
leave out information that could be valuable to the 
unsuccessful offeror. 

17. Question 17 This question was designed to determine 
if the unsuccessful offerors are satisfied with the debriefing 
on the technical, management, and cost portions of their 
proposal. 

a. Question to unsuccessful offerors 


I am satisfied with the debriefing on the 
technical, management, and cost elements of my proposal. 


a. always 
b. sometimes 
Cc never 
1. Discussion 
Forty one (41) unsuccessful offerors answered 
this question. Five (5) percent said that they are always 
Satisfied with the debriefing on the technical, management, 
and cost elements of their proposal. Eighty (80) percent 


stated that they are sometimes satisfied and 15% responded 


that they are never satisfied with the debriefing on these 
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elements. Three offerors commented that debriefings are 
generic in nature and not enough detail is given. 
b. Question to AMC 


Unsuccessful offerors are satisfied with the 
debriefing on the technical, management, and cost elements of 
their proposal. 


always 
sometimes 
never 
don’t know 


aaon 


1. Discussion 

Twenty nine (29) contracting officers answered 
this question. Twenty four (24) percent stated that 
unsuccessful offerors are always satisfied with the debriefing 
on the technical, management, and cost elements of their 
proposal. Sixty six (66) percent said that offerors are 
sometimes satisfied. Three (3) percent said they are never 
Satisfied and 7% stated that they didn’t know. Several 
comments were received. Three contracting officers stated 
that the unsuccessful offerors usually feel the debriefing is 
not forthcoming in this regard because the offerors want to 
know more about how they compared with the winner’s proposal. 
One contracting officer said that often times offeror’s 
engineers take the debrief on the technical aspects of the 
proposal as a personal attack and this makes it extremely 


difficult to debrief that portion. 
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C. Analysis 

A small percentage of both groups surveyed stated 
that unsuccessful offerors are always satisfied with the 
discussion on the technical, management, and cost portions of 
the debriefing. A large portion of each said offerors are 
sometimes satisfied. This is most likely due to two reasons. 
First, the offerors are looking for all the information they 
can get concerning the evaluation of these areas of their 
proposal and other proposals, especially the winners. The FAR 
and AFARS specifically deny access to some of the information 
that the contractors would like to see. Second, the 
debriefing may not contain all the information that the source 
selection committee has on the proposal. This could be due to 
poor documentation, a briefing from a second hand source who 
is not as knowledgeable as the evaluator, or a fear of 
debriefing something that was discovered and not addressed 
with the offeror during discussions. 

18. Question 18 The intent of this question is to 
determine if unsuccessful offerors get information which they 
can use in drafting proposals for future procurements. 

a. Question to unsuccessful offerors 


Debriefings give me information that leads to more 
competitive proposals in future Government competition. 


always 


a. 
b. sometimes 
C never 
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1. Discussion 
Forty one (41) offerors answered this 
question. Seven (7) percent stated that debriefings always 
give them information which leads to more competitive 
proposals in future procurements. Seventy one (71) percent 
reported that they sometimes get information to aid future 
proposals and 22% said they never get information which 
improves the competitiveness of future proposals. Two 
comments were submitted for this question. One contractor 
wrote that he rarely gets this information from the debriefing 
but through "contacts" in the community. The other comment 
was from an offeror who said that the only useful information 
he has gotten from the debriefings is that the low cost 
proposal always seems to win. As a result he is turning his 
best value proposals into low cost proposals. 
b. Question to AMC 
Debriefings give unsuccessful offerors information 
that leads to more competitive proposals for future Government 
competition. 
a. always 
b. sometimes 
C. never 
de don’t know 
1. Discussion 
Thirty (30) contracting officers answered this 


question. Twenty three (23) percent stated they believe 


debriefings always give information to unsuccessful offerors 
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that aid them in preparing more competitive proposals in 
future procurements. Fifty seven (57) percent Said 
debriefings sometimes give this information. Three (3) 
percent think debriefings never give offerors information that 
is useful for future procurements, and 17% don’t know. Three 
different comments were submitted. Six contracting officers 
stated that their intent is to give out information that will 
be useful for future competition. One official wrote that the 
offerors learn how AMC utilizes and evaluates best value 
procurement. If they don’t learn anything else they learn 
this. The last person to comment said that some contractors 
accept the critique and criticism of their proposal and learn 
from the debrief and some argue each point made and thus are 
too busy arguing and defending their position to learn 
anything. 
c. Analysis 
The majority of those surveyed on both sides 

indicated that debriefings sometimes provide information that 
leads to more competitive proposals in the future. The goal 
would be to increase the information that allows unsuccessful 
offerors to prepare more competitive proposals in the future. 
There are some road blocks that must be removed to improve 
this. Unsuccessful offerors must not allow their emotions or 


pride to interfere with assimilating the information given out 
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in the debriefing. Arguing each point brought up by the 
contracting officer detracts from good listening and may put 
the contracting officer on the defensive and cause him to 
refrain from passing along all the data he has. Conversely 
contracting officers must make every effort to get as much 
information to the contractor as they can. 

19. Question 19 This question was designed to determine 
if after a debriefing unsuccessful offerors fully understand 
why they do not have the most advantageous proposal. 

a. Question to unsuccessful offerors 


Upon conclusion of a debrief, I completely 
understood why my company did not win the contract award. 


a. always 
b. sometimes 
C never, why not 
1. Discussion 
All 42 offerors answered this question. Two 
(2) percent of the contractors reported that they always 
completely understood why they didn't win the contract award. 
Seventy six (76) percent stated that they sometimes completely 
understood and 22% said that they never completely understood 
why they didn't win the contract award. Several comments were 
received for this question. Five offerors commented that 
there is not enough information given out to completely 


understand why they didn't win the contract award. Three 


contractors stated that they often feel that the real reasons 
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why they didn't win the contract award are not given. Another 
offeror said that the Army doesn't compare proposals so you 
don’t know if you lost on technical merit or a subjective best 
buy. 
b. Question to AMC 
Upon conclusion of a debrief the unsuccessful 


offeror completely understands why he didn’t win the contract 
award j 


a. always 

b. sometimes 
c. never 

d. don’t know 


1. Discussion 

Thirty one (31) contracting officers answered 
this question. Twenty six (26) percent said they believe that 
after a debriefing contractors completely understand why they 
didn’t win the contract award. Forty two (42) percent stated 
they believe contractors sometimes completely understand. 
Three (3) percent responded that contractors never understand 
and 29% said that they don’t know if a contractor completely 
understands why he didn't win the contract award. Several 
comments were submitted. Two officials responded that a lack 
of details of the winner's proposal or any comparative 
rankings make it almost impossible for the offerors to 
completely understand why they lost. Three people wrote that 
some offerors can’t or refuse to recognize that someone else’s 


proposal could be better and thus can’t understand how they 
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lost. One contracting officer commented that the regulatory 
restrictions on what can be discussed sometimes prevents the 
offeror from getting enough information to completely 
understand why they lost. Finally another contracting officer 
replied that when the offeror’s proposal is very good and has 
no real weaknesses but the winning proposal is just a little 
better it may be very difficult for the offeror to understand 
why they didn’t win. 
c. Analysis 

Contracting officers believe the unsuccessful 
offerors understand why they lost the contract award at a 
higher rate than the unsuccessful offerors actually 
understand. This leads to the conclusion that contracting 
officers think the debriefings are more effective than they 
actually are. Two key problems are identified here. First, 
unsuccessful offerors state that not enough information is 
given out to understand why they didn’t win the contract 
award. Second, contracting officers commented that since 
comparisons between proposals can’t be made it makes it 
difficult for contractors to get enough information to 
completely understand why they lost. The Army AFARS thus has 
a negative impact on the quantity of information the 


unsuccessful offeror can get. 
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20. Question 20. The intent of this question is to 
determine if debriefings are of value to the unsuccessful 
offerors. 

a. Question to unsuccessful offerors 
The debriefings are to my company. 
valuable 


somewhat valuable 
not at all valuable 


QU 


1. Discussion 

Forty two (42) unsuccessful offerors answered 
this question. Thirty eight (38) percent stated that the 
debriefings are valuable, 50% said they are somewhat valuable 
and 12% reported that debriefings are not at all valuable. 
Contractors submitted four comments to this question. One 
contractor wrote that although debriefings are only somewhat 
valuable he never passes up an opportunity to meet with his 
customer. He continued that team building and getting to know 
the key players are too important to not attend a debriefing. 
Another offeror stated that the real details on the evaluation 
of his proposal are dug out over time from the command, thus 
making the debriefing not as valuable as it could be. Another 
comment was that there is nothing given to help with future 
proposals. Finally one unsuccessful offeror responded that 
the debriefings are cordial but not meaningful in terms of 


useful information given out. 
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a ee 
eS 


b. Question to AMC 


The debriefings I give are to unsuccessful 
offerors. 


valuable 
somewhat valuable 
not at all valuable 


aY 


1. Discussion 

Twenty nine (29) contracting officers answered 
this question. Sixty six (66) percent believe debriefings are 
valuable to unsuccessful offerors. Thirty four (34) percent 
believe debriefings are somewhat valuable to them and 3% 
believe debriefings are not at all valuable to unsuccessful 
offerors. Four comments were received with the answers. One 
contracting officer stated that since he can't compare scores 
and assessments his debriefings are not as useful to 
unsuccessful offerors as the offerors think they could be. 
Another contracting officer said that he is convinced 
debriefings are worthless because they don’t tell the 
unsuccessful offerors what they really want to know- how they 
stood compared to the winner. The next comment was that the 
regulation and legal constraints sanitize the debriefing so 
much that very little of any value is left in the debriefing. 
Finally, one official responded that all the unsuccessful 
offerors want to talk about is the winner’s proposal and he 


soes t provide much information on it. 
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C. Analysis 
Contracting officers believe debriefings are 
valuable to unsuccessful offerors at a much higher percentage 
than unsuccessful offerors say they are. Thus debriefings are 
not as valuable as contracting officers think they are. 
Consequently many contracting officers may not be making any 
attempts to improve the debriefing process as they believe the 
system is satisfying the needs of the unsuccessful offerors. 
On the other hand the contractors need to understand that some 
information that they would like to get debriefed on such as 
details of the winner’s proposal is not releasable under the 

current regulations. 

21. Question 21 This question was designed to determine 
if AMC offers to give debriefings to unsuccessful offerors 
that don’t request a debriefing on their own. 

a. Question to unsuccessful offerors 


The Army offers to debrief me even when I don’t 
ask for a debriefing. 


a. always 
b. sometimes 
E never 
1. Discussion 
Forty two (42) unsuccessful offerors answered 
this question. Twelve (12) percent stated that the Army 


offered to debrief them even when they didn’t ask for a 


debriefing. Thirty eight (38) percent said that the Army 
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sometimes asked them, and 50% responded that the Army never 
asked them. One contractor commented that the notification he 
receives that he is an unsuccessful offeror typically offers 
the opportunity for a debriefing. 

b. Question to AMC 


My organization offers to debrief all unsuccessful 
offerors, even those that don’t request a debriefing. 


a. always 
b. sometimes 
E never 
1. Discussion 
Thirty one (31) contracting officers answered 
this question. Fifty two (52) percent responded that they 
always offer to debrief all unsuccessful offerors. Thirteen 
(13) percent stated that they sometimes offer to debrief those 
offerors who don’t ask for a debriefing and 35% answered they 
never offer to debrief offerors that don’t ask for a 
debriefing. Several comments were given for this question. 
Six contracting officers stated that they never offer a 
debriefing but only honor requests. One official commented 
that at the major system level debriefings are normally pre- 
scheduled for all offerors. Another contracting officer 
commented he offers the debrief in the RFP but otherwise it is 
not mentioned. Finally, one official responded that he offers 
a debriefing in the letter notifying the contractor that he 


didn't win the contract award. 
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C. Analysis 

The answers between the two groups surveyed vary. 
The percentage of contracting officers who say they offer 
debriefings to all unsuccessful offerors, even those that 
Gon’t ask for one, is higher than the percentage of 
unsuccessful offerors who say the same thing. The key here is 
that offering debriefings to all unsuccessful offerors is not 
happening. The regulations do not require AMC to offer 
debriefings to anyone. It only states that debriefings will 
be given to those unsuccessful offerors that ask for one in 
writing. However, if debriefings are not offered there may be 
offerors who do not know they exist or that they can get one 
for the particular procurement they submitted a proposal for. 
22. Question 22 This question was designed around the 
specific language of H.R. 3161. The intent was to determine 
which points of the resolution unsuccessful offerors and 
contracting officers like and to determine how open each of 
the two groups are to liberalizing what can be discussed at 
debriefings. Both surveys asked for the respondents to circle 

all answers they agreed with. 

a. Question to unsuccessful offerors 


Debriefings should address (circle all that 


apply) , aS long as confidential business information is 
not disclosed. 

a. the basic proposed technical solution of the 
awardee 


92 





B. the overall evaluated cost of the awardee and 
debriefed offeror 

cC. cost or price associated with the major components 
of the awardee’s proposal 

d. overall ranking and total evaluation scores of the 
awardee and debriefed offeror 

e. technical point scores of the awardee and 
debriefed offeror 

f. none of the above 

g. other 

1. Discussion 
The unsuccessful offerors showed overwhelming 

support for answers a through e. They felt that all the 
changes proposed by H.R. 3161 were good ideas. Eighty three 
(83) percent of the offerors stated that debriefings should 
address the basic proposed technical solution of the awardee. 
Seventy nine (79) percent of the officials said debriefings 
should cover the overall evaluated cost of the awardee and 
debriefed offeror. Seventy six (76) percent of the 
contractors responded that the cost or price associated with 
the major components of the awardee’s proposal should be 
addressed in debriefings. Ninety (90) percent of the 
unsuccessful offerors stated that the overall ranking and 
total evaluation scores of the awardee and debriefed offeror 
should be covered. Eighty six (86) percent of the contractors 
said debriefings should contain the technical point scores of 
the awardee and debriefed offeror. Two (2) percent of the 


offerors stated that debriefings should contain none of the 


above. Several comments were submitted with these answers. 
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Three unsuccessful offerors stated that as much information as 
can be disclosed without violating proprietary rights should 
be given out. Another contractor responded that a narrative 
evaluation of the debriefed offeror's proposal should be given 
in addition to the other answers. One offeror said that the 
basic rationale for selection of the winning proposal should 
be given. Lastly, one contractor stated that more comparative 
information should be given out. 
b. Question to AMC 
Debriefings should address (circle all that 


apply) , as long as confidential business information is 
not disclosed. 





a. the basic proposed technical solution of the 
awardee 

b. the overall evaluated cost of the awardee and 
debriefed offeror 

C. cost or price associated with the major components 
of the awardee’s proposal 

d. overall ranking and total evaluation scores of the 
awardee and debriefed offeror 

e. technical point scores of the awardee and 
debriefed offeror 

f. none of the above 

g. other 


1. Discussion 
| Contracting officers didn’t show much support 
for including the information in these answers in debriefings. 
Thirteen (13) percent of the contracting officers stated that 
debriefings should address the basic proposed technical 
solution of the awardee. Forty one (41) percent of the 


officials said the overall evaluated cost of the awardee and 


94 


debriefed offeror should be included. The cost or price 
associated with the major components of the awardee's proposal 
should be covered by the debriefing according to 6% of the 
contracting officers. Twenty two (22) percent of these 
officials said that the overall ranking and total evaluation 
scores of the awardee and debriefed offeror should be covered 
in debriefings. Technical point scores of the awardee and 
debriefed offeror should be covered in debriefings according 
to 3% of the contracting officers surveyed. Twenty eight (28) 
percent of the contracting officers responded that none of the 
above should be covered. Sixteen (16) percent of the 
contracting officers stated that debriefings should contain 
Other items. All but one of these other answers stated in 
general terms that only the offeror’s proposal should be 
covered and that weak and deficient areas should be addressed. 
In other words these officials restated what the FAR requires. 
The other item listed was that the basis for why the award was 
given to the winner should be discussed. Two different 
comments were included with the answers. Two contracting 
officers said that information regarding the winner’s proposal 
Should be given only in general terms. Another contracting 
officer stated that he generally gives factor ratings and 
evaluated cost for the winning proposal and factor, sub-factor 


ratings and evaluated cost for the offeror’s proposal. 
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C. Analysis 

The unsuccessful offerors showed strong support 
for debriefings to include all the items that H.R. 3161 
proposed. Contracting officers, on the other hand, were much 
more conservative and only one of the proposed items received 
higher than a 22% support of the contracting officers. This 
item was revealing the overall evaluated cost for the 
offeror’s and winner’s proposals. Almost one third of the 
contracting officers stated that none of the answers should be 
addressed in debriefings. The answers given to this question 
show that unsuccessful offerors would like debriefings to 
include more specific information. Additionally this question 
shows the conservative nature of contracting officers with 
regard to debriefing. Contracting officers didn’t show strong 
support for any of these proposed additions to what should be 
covered in debriefings. 

238 Question 23 The intent of this question is to 
determine if when a debriefing is conducted has any impact on 
if an unsuccessful offeror will protest the contract award. 

a. Question to unsuccessful offerors 


The sooner I’m debriefed the less likely I am to 
protest. 


always 


a. 
b. sometimes 
E never 
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1. Discussion 
Forty one (41) unsuccessful offerors answered 
this question. Twelve (12) percent answered it is always true 
that the sooner they are debriefed the less likely they are to 
protest. Fifty one (51) percent stated that sometimes the 
sooner they are debriefed the less likely they are to protest. 
Fifteen (15) percent of the offerors stated that this is never 
the case. Twenty two (22) percent didn't answer one of the 
possible answers but commented the question was not applicable 
to them. This may mean they never protest or that the 
timeliness of a debriefing doesn't impact on their decision to 
protest. One offeror commented that the quality of the 
debriefing and not the time when it is given is the 
determining factor if a protest will be filed. Three offerors 

rate” they never protest. 

b. Question to AMC 


The sooner a debriefing is conducted, the more 
likely it is to prevent a protest. 


a. always 
b. sometimes 
C never 
1. Discussion 
Twenty eight (28) contracting officers 
answered this question. Thirty two (32) percent stated that 


the sooner a debriefing is conducted the more likely it is to 


prevent a protest. Fifty seven (57) percent said that this is 
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sometimes the case and 10% responded that this is never the 
case. Several comments were submitted with the answers. Four 
contracting officers commented that they don't think when a 
debriefing is done influences if an unsuccessful offeror will 
protest. Another contracting officer stated that the 
unsuccessful offeror only has 10 days to file a protest to 
stop the procurement action. 
C. Analysis 
Many unsuccessful offerors and contracting 
officers say that the sooner a debriefing is conducted the 
less likely the unsuccessful offeror is to protest. This 
should certainly be true for debriefings that are done within 
the window authorized for the unsuccessful offeror to submit 
a protest. A competent debriefing conducted within this 
window that shows the source selection process was proper and 
fair should yield few if any protests. A debrief conducted 
after the 10 day protest window may initiate a protest because 
the offeror may have questions concerning the fairness of the 
source selection process but doesn't have the information that 
the debriefing would provide that shows the selection process 
was proper. 
24. Question 24 The intent of this question is to 
determine what the strengths of the Army debriefing process 


are. This question is valuable because the set up and content 
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of a debriefing are very dependent on which command the 
debriefing is done in and which contracting officer conducts 
the debriefing. Debriefings in each command can be improved 
by incorporating the strengths from the other commands. 
a. Question to unsuccessful offerors 
List 3 strengths of the Army debriefing process. 
1. Discussion 
Twenty six (26) of the 42 unsuccessful 
offerors who returned surveys answered this question. Many 
strengths of the debriefing process were identified. The major 
themes from these strengths are listed below. The number of 
unsuccessful offerors that identified each area as a strength 
are annotated in parenthesis. 
èe Contracting officers offer to debrief unsuccessful 
SErerors (2). 
e There are no restrictions on who may attend (2). 
e The debriefings are done in a timely manner (8). 
e The evaluators are normally at the debriefing and they 
os their part or are available to answer questions 


e Debriefings are well organized and generally follow the 
RFP outline (2). 


e Debriefings allow for interaction between the two parties 
and both sides get to know each other (2). 


e Personnel who debrief are always prepared, many times with 
a Script (2). 
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0 Debriefings isolate major shortcomings and weaknesses in 
each area evaluated (4). 


è Strengths of our proposal are identified along with our 
weaknesses (2). 


o If the debriefing is conducted properly it will help 
improve the competitiveness of future proposals (1). 


® Debriefings are thorough (2). 


b. Question to AMC 
List 3 strengths of the Army debriefing process. 
1. Discussion 
Twenty one (21) contracting officers answered 
this question. Many strengths of the debriefing process were 
identified by the contracting officers. The major themes from 
these strengths follow: 
® Debriefings provide an opportunity for contracting 
officers to give information to unsuccessful offerors that 
will strengthen their future proposals (8). 
® Debriefings are given in a timely manner (4). 


® Debriefings show the fairness and integrity of the source 
selection process (6). 


® Debriefings add accountability to the evaluation process 
A 


0 Debriefings answer all questions relative to the offeror's 
proposal (1). 


0 Debriefings reveal the strengths and weaknesses of the 
offeror's proposal (5). 


® Debriefings give a lot of details to the unsuccessful 
offeror regarding the evaluation of his proposal (4). 
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è Contracting officers are willing to discuss all issues 
pertinent to the evaluation of the proposal (1). 


® Debriefings explain the basis for the award (1). 


C. Analysis 

The survey results show that unsuccessful offerors 
and contracting officers believe there are many strengths to 
the debriefing process. All the items given as strengths 
should benefit the debriefing process. The majority of the 
strengths listed were mentioned by four or fewer survey 
participants. This could indicate that debriefings are 
conducted somewhat differently by different commands and 
contracting officers and that the parties involved in the 
debriefing process have differences of opinion concerning what 
is important to the debriefing process. 

25. Question 25 This question was designed to identify 
the weaknesses of the debriefing process. Once these 
weaknesses are identified steps can be taken to correct these 
problems. This will improve the debriefing process. 

a. Question to unsuccessful offerors 
List 3 weaknesses of the Army debriefing process 
1. Discussion 
Thirty (30) unsuccessful offerors answered 
this question. Many weaknesses were listed. The major themes 


of the weaknesses follow: 
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® Debriefings are not done in a timely manner (5). 
e Debriefings do not provide enough details (9). 


e Personnel who debrief are very cautious in what they say 
(4). 


O Limited information is given out due to the restrictions 
imposed by the regulations (6). 


e Debriefings offer very little information useful for 
future proposals (3). 


e No information is given on the awardee's proposal (3). 
e The main players in the source selection are sometimes not 
in attendance (2). 
b. Question to AMC 


List 3 weaknesses of the Army debriefing process. 


1. Discussion 
Seventeen (17) contracting officers answered 
this question. They stated there were many weaknesses in the 
debriefing process. The major weaknesses they indicated 
follow: 
© Comparisons of proposals must be done to make the 
debriefing meaningful (3). 


O There are limits on the release of information due to the 
regulations (2). 


è Debriefing can’t cover any aspects of the winner’s 
proposal except his total price (2). 


èe Fear of protests causes contracting officers to control 


what is said in the debrief. Sometimes the debrief is 
even scripted (3). 
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O Key players in the source selection process are sometimes 
not available for the debriefing (1). 


v 3 more is revealed in a debriefing than is required 
lijis 
C. Analysis 
The research shows there are several weaknesses to 
the debriefing process. Every one of the items identified as 
a weakness should have an adverse impact on the debriefing 
process. All but two of the weaknesses given involved the 
lack of information given out in debriefings. This indicates 
that a major deficiency with the debriefing process is the 
lack of information released to unsuccessful offerors. 
26. Question 26 
a. Question to unsuccessful offerors The intent of 
this question is to determine if unsuccessful offerors ask a 
core set of similar questions at debriefings. If they do, 
contracting officers can prepare for these questions and 
unsuccessful offerors may then get better answers to these 
common questions. 


The 3 most common questions I ask during a 
debriefing are . 


1. Discussion 
Thirty three (33) unsuccessful offerors 
answered this question. A total of 74 questions were given. 


Many respondents listed only one or two questions for their 
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answer. Questions that were listed at least three times are 
considered common questions. The following nine questions 


appeared at least three times. 


e What was the evaluation of our technical approach (12)? 
e What are the weaknesses in our proposal (8)? 
è What was the evaluation of our cost (8)? 


e What was the winner's proposal and evaluation of that 
proposal (6)? 


e How did we compare with the awardee (5)? 

e What are the strengths of our proposal (4)? 

eo How did the evaluators judge our risk (3)? 

e What was our overall ranking (3)? 

e What should have I done to have won this competition (3)? 

2. Analysis 
There are many similar questions that are 

asked by unsuccessful offerors during debriefings. One of 
these questions cannot be answered according to the guidance 
in the AFARS and another question can only be answered ina 
very general nature in accordance with the FAR. The AFARS 
states that comparisons will not be made between proposals [1: 
p.10-3]. Therefore unsuccessful offerors will not get an 
answer to their question of how did they compare to the winner 
unless the contracting officer deviates from the AFARS 


guidance. The FAR states that debriefings will not reveal the 
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relative merits or information covered by the Freedom of 
Information Act on competitors proposals [7: p.15-41]. This 
will severely limit what can be discussed in response to the 
question what was the winner’s proposal and evaluation of that 
proposal. The other questions can be thoroughly answered by 
debriefing personnel. Contracting officers that are prepared 
to answer these questions will give better answers to these 
questions than those who are not ready to answer them. 

E. Question to AMC This question was designed to 
determine if AMC MSC's have internal instructions that give 
guidance to contracting officers on debriefing unsuccessful 
ST ferors. 


My organization has internal instructions covering 
debriefing procedures. 


a. yes 
ORTO 


l1. Discussion 
Thirty one (31) contracting officers answered 
this question. Forty five (45) percent stated that their 
organization has internal instructions covering debriefing 
procedures. Fifty five (55) percent of the contracting 
officers responded their command doesn't have internal 


instructions on debriefing procedures. 
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2. Analysis 

Results of this question show that not all 
commands have guidance for debriefing unsuccessful offerors. 
Lack of any clear guidance as to what may be covered in a 
debriefing and what may not can cause contracting officers to 
take a very conservative view of the general guidance in the 
FAR and AFARS. Follow up telephone calls to the five MSC’s 
surveyed revealed that three MSC’s have written guidance 
concerning debriefings. Only one of these has any specific 
guidelines. This MSC’s internal instructions has several 
recommendations for what should be covered in a debriefing. 
It states that the contractor officer should consider the 
following items in the debriefing: a general explanation of 
the source selection organization, the basis for the award, 
the ratings the offeror's proposal received on all the factors 
and sub-factors, an explanation of why each rating was given, 
the advantages and disadvantages of the proposal, and a brief 
explanation of why the successful offeror won. The 
explanation of why the winner won could include showing the 
winner’s ratings for each factor and sub-factor without any 
detailed explanation of each rating. This guidance adds some 
structure to the debriefing process. It also provides some 
framework for what can be discussed about the winner’s 


proposal evaluation. This can add valuable information to the 
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debriefing, as the unsuccessful offeror can see where his 
proposal stood in relation to the winner's in terms of the 
overall rating for each area. The guidance this MSC’s 
internal instructions provide allow contracting officers who 
have interpreted the FAR and AFARS conservatively to give 
information that they may not have given absent this guidance. 

27. Question 27 The intent of this question was to have 
both the unsuccessful offerors and contracting officers offer 
their ideas on things the Army could do to improve the 
debriefing process. 

a. Question to unsuccessful offerors 


The Army could improve the debriefing process 


by 
1. Discussion 
Twenty eight (28) unsuccessful offerors 
answered this question. Many proposals were provided to 
improve the debriefing process. The major themes of these 


proposals follow: 


© Allow some comparisons with the winning proposal (7). 


© Improve evaluation process so that the people debriefing 
can give an honest and detailed debriefing (5). 


O Be more specific when discussing deficiencies (4). 
® Provide more information on winning proposal (4). 
® Approach process in a more positive way (3). 


O Provide read ahead packets (2). 


107 


e Debriefings should be done in a more timely manner (2). 
è Discuss strengths as well as weaknesses (2). 


e Allow contractors to submit questions before debriefing so 
answers can be better researched (1). 


è Reveal how unsuccessful offeror ranked in all areas 
(1). 


èe Provide rating and risk assessment for each evaluated area 
for both the unsuccessful offeror’s and winner’s proposal 
(195 


O Release SSA decision document (1). 


b. Question to AMC 


I/The Army could improve the debriefing process by 


1. Discussion 
Twenty two (22) contracting officers answered 
the question. Many answers were given. The major themes for 


improving the debriefing process follow: 


e Issue some direction other than what is in the FAR (4). 

O Write solicitations and evaluation criteria as precise as 
possible to make evaluations and subsequent debriefings 
easier (2). 

e Ask for questions in advance (1). 


® Give unsuccessful offerors their evaluation ratings 
(1). 


® Establish guidelines for consistency of releasable 
information (1). 


® Allow comparisons between unsuccessful offeror’s proposal 
and winning proposal (2). 
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O Give unsuccessful offerors debriefing charts in advance of 
the debriefing (1). 


O Provide maximum information allowed (3). 


e Contracting officers should be more open and honest 


a. 


0 Give unsuccessful offerors source selection decision 
memorandum (1). 


O Hold debriefings later than they are now held (2). 


C. Analysis 

The results of the research show there are many 
ways that contracting officers and AMC could improve the 
debriefing process. Several of the recommended ways to 
improve debriefings were mentioned by both unsuccessful 
offerors and AMC officials. These areas were allowing some 
comparisons with the winning proposal, give out debriefing 
packets and do so before the debriefing, provide offerors with 
their ratings for the evaluated areas, provide as much 
information as possible, and releasing the source selection 
decision document. One recommendation that the two survey 
groups disagreed on was when to conduct debriefings. 
Contractors stated that the debriefing process could be 
improved by holding debriefings in a more timely manner while 
AMC officials said to improve the debriefings they should be 


held later than they are now held. 
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28. Question 28. This question was designed to get 
recommendations from unsuccessful offerors and contracting 
officers of how unsuccessful offerors could improve the 
debriefing process. 

a. Question to unsuccessful offerors 


I/unsuccessful offerors could improve’ the 
debriefing process by a 


1. Discussion 
Twenty four (24) unsuccessful offerors 
answered this question. The major themes of their proposals 


follow: 


O Submit questions before the debriefing (7). 
O Have the right personnel there (3). 
® Approach the debriefing with a positive attitude (5). 


O Limit protests to procurements that show a clear 
impropriety or prejudice (2). 


O Ask as many questions as needed but ensure they are 
relevant and appropriate (6). 
b. Question to AMC 


Unsuccessful offerors could improve the debriefing 
process by a 


1. Discussion 
Twenty one (21) contracting officers answered 
the question. Several themes were given as possible ways to 


improve the system: 
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e Accept source selection and don't protest unless 
impartiality is displayed (3). 


O Be receptive rather than defensive (2). 
è Don’t challenge the source selection decision (3). 


O Personnel in attendance should have a good understanding 
of evaluation factors and proposal (2). 


è Unsuccessful offerors should ask questions in advance 


ae 


èe Unsuccessful offerors should ensure their decision makers 
for technical and cost areas are in attendance (3). 


è Unsuccessful offerors should be familiar with regulatory 
restrictions concerning debriefings (1). 


@® Unsuccessful offerors should not be so emotional (2). 
c. Analysis 
The results of the Survey indicate there are 
several ways that unsuccessful offerors could improve the 
debriefing process. Four of these recommendations were made 
by both groups of respondents. They are: submit questions 
before the debriefing, have the right personnel attend the 
debriefing, approach the debriefing with the right frame of 
mind, and limit protests to procurements that show a clear 
impropriety or unfairness in the source selection process. 
Two of these recommendations could be difficult for all 
contractors to execute. Approaching a debriefing with the 
right frame of mind by being receptive rather than defensive 


and accepting, not challenging the source selection can be 
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hard to do. It is natural for a contractor to be defensive 
when his proposal was not selected as the winning proposal and 
then attend a debriefing where someone tells him all the 
things that were wrong with the proposal. It is also natural 
for contractors to think they submitted the best proposal and 
the only way they could have lost was if the source selection 


team somehow made a mistake. 


C. INTERVIEWS 

1. General 
Interviews were done with defense contractors and 
contacting officers as a follow up to the surveys. The 
interviews were done with three defense contractors to get 
their opinions on how the debriefing process could be 
improved. These interview participants included a large 
hardware company, a large services company and a small 
hardware company. The intent of the interviews was to have a 
discussion and ask questions to illicit responses on how to 
improve the debriefing process that may have been missed by 
the survey participants. Contracting officers from three 
MSC’s within AMC were then interviewed to get their opinions 
on the feasibility of implementing the major suggested changes 


to the debriefing process. 
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2. Interview Results 

The three defense contractors suggested a total of 19 
changes to the current debriefing process that they felt would 
improve debriefings. All but four of these changes were 
identified by the survey participants. The contracting 
officers that were interviewed believe 14 of these 
recommendations could be implemented. 

Recommendations which were identified in both the 
Surveys and interviews that would improve the debriefing 


process are: 


O Debriefings should be done in a timely manner. 


e AMC MSC's should not place any limits on the employees 
that may attend the debriefing from the unsuccessful 
offerors. 


e Contracting officers should debrief in an outline similar 
to the RFP. 


e Contracting officers should give unsuccessful offerors a 
debriefing packet and do so before the debrief if 
possible. 


è Unsuccessful offerors should submit questions in advance 
of the debriefing. 


O SSEB members and any other evaluators should be in 
attendance at the debriefing to participate in the debrief 
and answer questions. 


® Debriefings should cover both the strengths and weaknesses 
of the offeror’s proposal. 


® Debriefings should cover the ratings of the evaluated 
areas for both the offeror’s and winner’s proposals. 
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è Narrative comments should cover all evaluated areas and be 
specific. 


O General details of the winner's proposal evaluation should 
be discussed. 


e Some comparisons between unsuccessful offeror’s and 
winner’s proposals should be revealed. 


® Contracting officers should discuss the basis for the 
award decision. 


e Some of the restrictions that the regulations stipulate 
concerning what shouldn’t be revealed in debriefings 
should be lifted. 


è Specific guidance on what should and shouldn’t be covered 
in debriefings should be developed. 


Oo The SSA decision document should be released to 
unsuccessful offerors. 


Contracting officers believe that several of these 
recommendations are not feasible. They believe that handouts 
should be given out to the unsuccessful offerors. This will 
allow the offerors to listen more closely to the debrief 
rather than trying to write down what they hear and see. 
However it would be difficult to give these handouts to the 
unsuccessful offerors before the debriefing if the debriefings 
are conducted soon after the contract award. Some time is 
needed to prepare handouts for all the unsuccessful offerors 
and there would not be enough time to prepare all these 
handouts far enough in advance to send them to the offerors 


[22]. 
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Next, contracting officers believe that ratings for 
the winner's proposal should not be given out in the 
debriefing [21]. They feel that discussing the winner's 
ratings would provide too much information on the winner's 
proposal. 

Providing the general details of the winner's proposal 
evaluation to the unsuccessful offeror's is inappropriate 
according to contracting officers [22]. Some details on the 
winner's cost or price can be released but this is the only 
information that should be released. Contracting officers 
further stated that if unsuccessful offeror’s want any 
additional information on the winner’s proposal they should 
request it under the Freedom of Information Act [21]. 

Finally, contracting officers interviewed believe that 
specific guidance on what should and shouldn’t be covered in 
debriefings should not be developed. They feel they have the 
latitude to work within the spirit of the regulations to 
tailor debriefings as needed depending on the situation [21]. 

The remaining four recommendations were identified in 
the interviews only. The first of these recommendations is 
that debriefings should be offered to all unsuccessful 
Offerors [14]. Offering a debriefing shows courtesy to the 
unsuccessful offeror. The contractor spent precious resources 


on the proposal and offering to debrief him shows the 
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contracting command appreciates the offeror's attempt to do 
business with the command. Additionally the offer to debrief 
all offerors shows the contractors that the contracting 
officer has nothing to hide and that the source selection was 
done properly. Finally, offering a debriefing to all 
unsuccessful offerors will ensure that those contractors who 
are unaware that they are entitled to a debriefing know they 
can receive one. There is nothing that prevents contracting 
officers from offering to debrief all unsuccessful offerors 
and some contracting officers already do this [21]. 
Debriefing all evaluated areas would also improve the 
debriefing process [13]. A debriefing that covers all 
evaluated areas allows the unsuccessful offeror to understand 
where his proposal stood in relation to all evaluation 
criteria. This type of debriefing would be superior to one 
where the offeror was only debriefed in certain areas that he 
was particularly weak in, as covering all areas would provide 
a broader base of information about the entire proposal. 
Another defense contractor stated that not only should 
each evaluated area in the debrief be discussed but the sub- 
factors in each area should be briefed [13]. Debriefings that 
address each sub-factor area should be much more informative 
than debriefings that address only evaluation factors or 


general areas. Offerors would get an understanding of how 
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their proposal stood up to the evaluation criteria on a much 
more specific basis. 

Revealing the rankings for each evaluated area for 
both the unsuccessful offeror's and winner's proposals would 
also improve the debriefing process [14]. This would allow 
contractors to see where they stood in relation to the other 
offerors and the winner. This information cannot be released 
for technical areas, however, according to the FAR [7: p.15- 
41]. Additionally giving out this information for other areas 
could be considered making point by point comparisons, which 
the AFARS prohibits. Revealing the rankings for the 
unsuccessful offeror’s and winner’s evaluated areas would be 
inappropriate according to contracting officers. Contracting 
officers stated this would be providing point by point 


comparisons which is prohibited by the AFARS [22]. 


D. SUMMARY 
1. Debriefings within AMC 
The results of the survey provide an overview of what 
a debriefing is like within AMC. However, as the survey 
results show no two debriefings are exactly the same. There 
are too many variables that can be changed to effect the 
debriefing process. These variables include things such as 


who the contracting officer is, the size and type of 
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procurement, the number and complexity of evaluation criteria 
used, the detail of the evaluation, the experience of the 
personnel involved, and the concerns and questions of the 
unsuccessful offerors. This section summarizes the answers to 
the survey which show what a generic debriefing of an 
unsuccessful offeror is within AMC. It will discuss when and 
where done, who attends, and what is discussed. Because no 
two debriefings are the same, debriefings within AMC may vary 
somewhat from the following baseline description. 
Debriefings must be conducted for those unsuccessful 
offerors that request a debriefing in writing. This is in 
accordance with the FAR. The number of unsuccessful offerors 
that request a debriefing is generally quite high. More than 
two thirds request a debriefing with this number being even 
higher for major acquisitions or best value type procurements. 
In addition, many contracting officers offer to debrief the 


unsuccessful offerors even if the offeror does not ask fora 


debriefing. This is not required by any regulation but is 
done out of courtesy for the unsuccessful offerors. Many 
contractors will accept this offer of a debrief. This 


increases the percentage of contractors being debriefed to a 
number higher than those that would have been debriefed from 


a contractor initiated request. 
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Two regulations provide guidance on when debriefings 
will be conducted. The FAR states that offerors will be 
debriefed as soon as possible after their request. AMC P 715- 
3 Vol. 1 states that debriefings will be held after contract 
award [2: p.51]. Contracting officers hold debriefings as 
soon as possible after their request. Contracting officers 
report this is usually within ten days of the request. 
Unsuccessful offerors dispute this and report that as the 
majority say debriefings are held between 11 and 20 days of 
request. 

There is no written guidance on where debriefings are 
to be held. However, most debriefings are in person and it is 
standard practice that debriefings are held at the contracting 
officer’s command. If the contractor wants to attend a 
debriefing it is incumbent on him to travel to the contracting 
officer’s location. Sometimes a telephone debriefing may be 
conducted but this is done only at the unsuccessful offeror’s 
direction. Debriefing by phone is normally done when the 
contract is small or the debriefing is easily understandable 
by phone. In these cases it is not cost effective for a 
contractor to pay travel costs to attend an in person 
debriefing. 

The number of people who attend the debriefing is 


dependent on the size and complexity of the procurement. For 
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large or complex procurements representatives of AMC may 
include the source selection evaluation board chairman, the 
factor chairmen, the contracting officer, contracting 
specialist, and a legal representative. Any other special 
support personnel are added as needed. Representatives from 
the unsuccessful firm that may attend a debriefing for large 
or complex procurements may include the vice president, 
program manager, proposal manager, and contracts manager. For 
smaller or less complex procurements there would generally be 
fewer personnel from each side participating. This could go 
all the way done to a one on one situation where the 
contracting officer debriefs a contracting official from the 
unsuccessful firm. 

The debriefing may be conducted by the contracting 
official alone or he may be joined by other personnel, 
especially for large procurements. For these large 
procurement debriefings the contracting officer is responsible 
for the set up and organization of the debriefing. He or the 
source selection evaluation board chairman is usually the 
official that conducts the introduction and general portions 
of the debriefing. The source selection evaluation board 
factor chairmen or representatives will conduct a debrief on 
their portion of the source selection and answer any related 


questions. 
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The information that is given out in the debriefing is 
the most important aspect of the debriefing. Obtaining this 
information is, of course, why the unsuccessful offerors ask 
for the debriefing. They generally look for as much 
information as they can get to understand why they didn't win 
the contract award. Most AMC officials say they debrief in 
accordance with the RFP, although the high percentage that say 
they do is disputed by the unsuccessful offerors. The 
debriefing will address the technical, management and cost 
merits of the proposal and in most cases identify weaknesses 
in the offeror’s proposal. Debriefing officials may sometimes 
discuss the winner's proposal, but only in very general terms. 
Details of the winning proposal are not disclosed nor are 
comparisons made between the offeror's proposal and other 
offerors' proposals, to include the winner's proposal. In 
addition, proprietary business data from other offerors is 
never discussed. 

The unsuccessful offeror is permitted to ask questions 
at the debriefing. Questions pertaining to the offeror’s 
proposal are answered, although detailed or difficult 
questions may result in an answer being provided sometime 
after the conclusion of the debriefing. Questions pertaining 
to another offeror’s proposal are normally not answered. They 


May be answered if they are of only a very general nature, 
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result in no point by point comparisons and don't involve 
proprietary business information. 

2. Major Items Identified To Improve Debriefing Process 

The research shows there are many areas of the 

debriefing process that can be improved. Many areas for 

improvement have been mentioned several times by the 

respondents throughout the research effort. These areas are 


summarized here. 


® Debriefings should be conducted in a timely manner. 


® No limits should be placed on who the unsuccessful 
offerors send to debriefings. 


® Contracting officers should debrief in an outline similar 
to the RFP. 


® Debriefing packets should be provided to unsuccessful 
offerors and the packets should be given to them before 
the debriefing if possible. 


® Unsuccessful offerors should submit questions before the 
debriefing is conducted. 


® SSEB members and other evaluators should be available to 
participate in the debriefing. 


® The strengths as well as the weaknesses of the offeror’s 
proposal should be covered in debriefings. 


® The ratings of the offeror’s and winner’s proposal 
evaluations should be disclosed. 


® Narrative comments should be specific and be given for all 
evaluated areas. 


® General comments on the winner’s proposal evaluation 
should be discussed. 
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Some comparisons should be made between the winner's and 


unsuccessful 


offeror's proposals. 


The basis for the award decision should be revealed. 


Some regulatory restrictions should be lifted. 


Specific guidance should be developed which covers what 
should and shouldn’t be discussed in debriefings. 


Contracting officers should release the source selection 
decision document to unsuccessful offerors. 


Contracting 
unsuccessful 


Contracting 
areas. 


Debriefings 
level. 


Bontracting 
rankings for 


officers should offer to debrief all 
offerors. 


officers should debrief all evaluated 


should be done down to the  sub-factor 


officers should discuss the evaluation 
the unsuccessful offeror and winner. 


E. SUMMARY 
This chapter presented and analyzed the data obtained from 
the surveys and interviews conducted. The research showed 


there is no standard debriefing process but debriefings are 


dependent on several factors. The research also showed that 


there are many ways the debriefing process can be improved. 


Chapter V will present conclusions, make recommendations to 


improve the debriefing process, answer the research questions 


and make recommendations for further research. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. GENERAL 

There are several conclusions that can be made on the 
current debriefing process based on the discussion and 
analysis of the research done. Many recommendations can be 
made to improve the current debriefing process. This chapter 
will address the conclusions and recommendations on the 
debriefing process, answer the research questions and point 


out areas for further research. 


B. CONCLUSIONS 

There are three conclusions that can be made based on the 
research done. 

1. Debriefings are not standard 

The survey data shows that debriefings within AMC, 

although somewhat similar, are not standard. Rather 
debriefings are directly influenced by a variety of factors. 
The main factors include the command and contracting officer 
conducting the debriefing, and the size and type of 


procurement. 
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2. Current debriefing process is working 
The current debriefing process is working. The 
research shows that respondents get some value out of 
conducting and attending debriefings. However, the 
respondents feel that the current debriefing process is not 
working as well as it could and improvements could be made to 
the debriefing process. 
3. Improvements can be made to the current process 
The research shows that there are many changes that 
can be made to the current debriefing process that can improve 
debriefings. Changes Can be made by both unsuccessful 
offerors and contracting officers to improve the process. 
These changes can be made to the set up and format of the 


debriefings and also the debriefing content. 


C. RECOMMENDATIONS 

There are 13 recommendations that commie) made based on the 
research done. 

1. Offer debriefings to all unsuccessful offerors 

Contracting officers should offer debriefings to all 

unsuccessful offerors. At the point in the procurement 
process where the debriefing takes place the contracting 
officer is providing a service to the unsuccessful offerors. 


To provide this service to the maximum number of contractors 
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it would be best to offer debriefings to all unsuccessful 
offerors. The regulations do not require AMC to offer 
debriefings to anyone. They only state that debriefings must 
be given to those offerors that request a debriefing in 
writing. However offering debriefings to all unsuccessful 
offerors will ensure that contractors unaware of their chance 
to be debriefed or contractors who are unsure if they can get 
a debriefing in certain procurements, know they may be 
debriefed. Additionally, offering to debrief unsuccessful 
offerors shows courtesy for the efforts of all contractors 
attempting to win the contract award. Offering debriefings 
also shows that the contracting command has nothing to hide 
and that the source selection was done rationally and in 
accordance with the RFP. 
2. Debriefings should be timely 

Debriefings should be done ina timely manner. The 
results of the research show that 50% of all respondents 
believe debriefings should be done within 10 days after 
contract award. Therefore a timely debriefing can be 
considered one that is done within 10 days after contract 
award. Timely debriefings offer several benefits. First 
there is a better chance that SSEB members and other 
evaluators who can enhance the quality of the debriefing will 


be able to participate in the debriefing. The longer the time 
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period between the contract award and the debriefing the more 
likely it is that these personnel will have gone on to other 
business and thus not be available for the debriefing. 
Additionally, contractor personnel can get on with other 
business quicker when the debriefing is done in a timely 
manner. Debriefings that are done quickly can also boost the 
confidence that unsuccessful offerors have in the source 
selection process. By providing timely debriefings AMC 
officials show that they are organized, professional, and can 
debrief from the source selection evaluation documentation 
without needing extra time to prepare additional data to 
justify the contract award or proposal evaluation. Conducting 
debriefings as soon as possible and possibly even pre- 
scheduling them indicates that AMC officials believe 
debriefings are an important part of the source selection 
process and not just a necessary evil. Timely debriefings may 
also reduce the number of protests that are filed. A 
competent debriefing done within the 10 day protest window 
after contract award should show the contractor that his 
proposal received a fair evaluation and that the source 
selection decision was proper. A debriefing that is not done 
until after this window closes may prompt a protest before the 
protest window closes if the contractor has any doubts about 


the fairness of the selection decision. A source selection 
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that is protested is harder to debrief and provides less 
information than one that is not protested as the personnel 
doing the debriefing would be much more conservative with 
regard to what information they release. 
3. Place no limits on who may attend debriefings 
Contracting commands should not place any limits on 
who the unsuccessful offerors may send to debriefings. The 
unsuccessful offeror is in the best position to determine who 
he needs to send to the debriefing to get the information he 
desires. Placing limits on the number of personnel or type of 
employees he may send can have an adverse impact on what the 
contractor extracts from the debriefing. Placing limits on 
who can attend can also invite unwarranted conclusions from 
the unsuccessful offeror. He may feel that the contracting 
command is trying to hide something by limiting who may attend 
or he may feel that he would have gotten more from the 
debriefing if he could have sent all the personnel he wanted 
to send. AMC officials need to keep the debriefing process as 
credible as possible and allowing contractors to send who they 
want to send to the debriefing helps in this regard. Thus, 
AMC personnel should fight to overcome any obstacles that may 
place limitations on who unsuccessful offerors may send to 


debriefings. 
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4. Give contractors debriefing packets 

Contracting officers should give unsuccessful offerors 
debriefing packets and provide them before the debriefing if 
possible. Debriefing packets would improve the debriefing in 
several ways. First, contractors could shift their attention 
from the distracting task of copying charts and slides and 
taking notes to listening to the content of the debriefing and 
taking a few notes. Additionally, the unsuccessful offeror’s 
contracting officials will have an official record of the 
debriefing from which they can, in turn, debrief their 
superiors. They can also use it to refer to later if 
questions arise concerning the debriefing. Next, if the 
debriefing packet is given to the contractor before the 
debriefing it would allow the contractor to prepare for the 
debriefing. They could study the advance material and tailor 
their interest and questions to areas they don’t understand or 
agree with. Contractors could also tailor who they send to 
the debriefing based on the advance material they get. 

5. Evaluators should participate in debriefings 

The SSEB members and any other evaluators should 
participate in the debriefings. They have the expertise and 
intimate knowledge of the proposal evaluation to best debrief 
Or answer questions in their respective evaluation areas. 


Debriefings can be much more informative when evaluators are 
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available to participate in the debriefing. Since contracting 
officers do not have the same level of expertise or knowledge 
about the proposal evaluation, debriefings done solely by the 
contracting officer tend to be dry and uninformative and in 
many cases scripted. Additionally, the contracting officer 
would be hard pressed to give any data other than what is in 
the script as he would not have the capability to provide any 
more information. Therefore in an attempt to give as much 
information as possible to the unsuccessful offeror it is best 
to have the evaluation team available to participate in the 
debriefing. 
6. Submit questions before debriefings 

Unsuccessful offerors should submit questions before 
the debriefings are conducted. This will allow contracting 
officers to properly research these questions, prepare 
Satisfactory answers and present these answers in the 
debriefing. This will reduce the problem of asking a question 
during the debriefing and not getting an adequate answer 
because there was not enough time to gather the necessary 
information and prepare the answer. 

7. Brief in an outline similar to RFP 

The debriefing should be done in an outline similar to 

the RFP. Interview results show that a debriefing done in an 


outline similar to the RFP is easier to follow. The 
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contractor can go right down the RFP and his proposal as it is 
debriefed and follow the debriefing rather than flipping 
pages trying to stay with a debriefing that skips around. 
Additionally, according to one contracting officer, using an 
outline similar to the RFP makes it easier for the contracting 
officer to put together and format the debriefing while 
ensuring all relevant areas are covered. 
8. Cover strengths and weaknesses 

Debriefings should cover the strengths as well as the 
weaknesses of the offeror’s proposal. Noe only as it 
important to discuss where the unsuccessful offeror’s 
weaknesses are in their proposal but it is also important to 
discuss their strengths. Revealing the offeror’s strengths 
will let the contractor know where the contracting command 
believes the offeror is particularly strong. This will allow 
the contractor to not waste resources by making unnecessary 
modifications to his strengths and to capitalize on these 
strengths in future proposals. 

9. Give the offeror his ratings 

Contracting officers should give the unsuccessful 
offerors their ratings for each of the evaluated areas. The 
ratings, combined with good narrative comments, will give 
offerors a better understanding of how their proposal stood up 


to the evaluation compared to a debriefing where just comments 
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are given out. As an example an evaluated area that has three 
positive comments and one negative comment with a rating of 
outstanding would indicate that the offeror was strong in that 
area. The same comments with no rating would leave the 
offeror with a question of how strong his proposal was in that 
area. 
10. Debriefing should cover all evaluated areas 

The debriefing should cover all evaluated areas. 
Unsuccessful offerors should receive a better briefing if all 
evaluated areas are covered in the debriefing. The 
contracting officer and others who debrief are familiar with 
all evaluated areas and have the evaluation comments at their 
disposal. They should pass on their evaluation remarks for 
all areas. Telling an unsuccessful offeror that he was 
Satisfactory in an area and then following up with any 
comments noted from the evaluation is more informative than 
bypassing satisfactory areas and leaving it up to the 
contractor to guess how he did in the bypassed areas. 

11. Debrief down to sub-factor level 

Not only should personnel conducting the debriefing 
brief each evaluated area but they should brief down to the 
sub-factor level. In order to be specific enough for the 
remarks to be helpful and informative the debriefing should go 


to this level. A common complaint from the unsuccessful 
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offerors is that debriefings are too generic in nature. 
Briefing down to the sub-factor level with good narrative 
comments will help alleviate this problem. 
12. Use the latitude authorized to deviate from AFARS 
when in best interests of parties involved 
Contracting officers should use the latitude they have 
to deviate from the strict guidance of the AFARS when it is in 
the best interests of AMC to do so. A conservative 
interpretation of the AFARS is that no information should be 
given out on any competitor’s proposal or evaluation of their 
proposals and that no comparisons will be made between 
proposals. The research has shown that the authors of the 
AFARS intended to allow contracting commands to deviate from 
the guidance of the AFARS when it made sense to do so. 
Additionally, several commands routinely deviate from this 
guidance to get as much information to the unsuccessful 
offerors as they can. This includes giving out general 
details of the winner’s proposal and the evaluation done on it 
and making some general comparisons between the winner’s and 
unsuccessful offeror’s proposals. This additional information 
will help the unsuccessful offerors understand how their 
proposal stood up in a general nature to the winner’s 


proposal. 
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13. Release the SSA decision document 

Contracting officers should release the SSA decision 
document to unsuccessful offerors during the debriefing. 
Research has shown that releasing this document to 
unsuccessful offerors provides contractors with information 
they clearly want and is useful to them. It has also shown to 
add credibility to the source selection process in the eyes of 
the contractors. The document must be scrubbed to ensure any 
confidential business data is deleted and that any items 
expressly prohibited by the FAR are not included in the 
document that is released. This document should show the 
basis for the award decision. Items that could be released in 
this document include the ratings for each area for the 
winner’s and offeror’s proposals, some general comparisons and 
why the decision was made. Comments such as the following 
would be useful: Company A's proposal was good but not as 
advantageous to the Army as company B's proposal. Company B 
was Slightly better than company A in the technical area. 
Company A was better in the area of past performance. Company 
B was slightly better in the management area than company A. 
Company B’s advantages in the technical area and management 
more than offset company A’s advantage in past performance 


therefore company B was chosen for the contract award. 
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D. ANSWERS TO RESEARCH QUESTIONS 

ir, Primary. What are the key problems and issues 
associated with AMC's process of debriefing unsuccessful 
offerors and what can be done to improve the process? 

Key problems with the debriefing process include 
debriefings not being done in a timely manner, key personnel 
from AMC not attending the debriefing, limits being placed on 
who may attend debriefings from unsuccessful firms, and 
limited information being given out in the debriefings. 

The process can be improved by offering debriefings to all 
unsuccessful offerors, conducting debriefings in a timely 
manner, not limiting who may attend debriefings, providing 
unsuccessful offerors with debriefing packets, and providing 
more information to the unsuccessful offerors. 

2. Subsidiary. 

a) What is the current debriefing process used by 
AMC? 
There is no standard debriefing process used by 
AMC. Rather AMC MSC's and contracting officers tailor 
debriefings based on the guidance in the FAR, AFARS, internal 
operating instructions and situation. 
There are some generic procedures for AMC 
debriefings however. Debriefings must be conducted for all 


unsuccessful offerors that ask for a debriefing in writing. 
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The debriefings are held at the contracting command and 
several people representing the contractor and contracting 
command normally attend the debriefing. The contracting 
command will provide the unsuccessful offeror with information 
that will help the offeror determine how their proposal stood 
up to the evaluation criteria and why they did not win the 
contract award. 

b) What are the problems associated with AMC’s 
current debriefing process from AMC’s perspective? 

Several problems were identified with the 
debriefing process according to AMC contracting officers. 
First, sometimes the key players in the source selection 
process are not available to participate in the debriefing. 
Next, limited information is put out in the debriefing. Many 
debriefings do not cover all evaluated areas, give narrative 
comments for all areas, reveal ratings for the offeror’s 
evaluated areas, make any comments about the winner’s 
proposal, and make any comparisons between the winner’s and 
offeror’s proposals. Additionally debriefings are, at times, 
not timely. 

c) What are the problems associated with AMC's 
current debriefing process from the unsuccessful offeror’s 


perspective? 
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There are many problems with the debriefing 
process according to unsuccessful offerors. Debriefings are 
not done in a timely manner and do not provide enough details. 
Information is limited due to restrictions imposed by the 
regulations. Little or no information is given out on the 
winner’s proposal. Sometimes the main players in the source 
selection process are not in attendance at the debriefings. 

d) How can the debriefing process be modified to help 
AMC and the unsuccessful offerors? 

There are several ways to modify the debriefing 
process to help the parties involved. Debriefings should be 
offered to all unsuccessful offerors. Debriefings should be 
done in a timely manner and there should be no limits placed 
on who may attend the debriefings. Contracting commands 
should give unsuccessful offerors debriefing packets and 
evaluators should participate in the debriefings. 
Unsuccessful offerors should submit questions before the 
debriefing. The debriefing should be conducted in an outline 
Similar to the RFP. More information should be given out in 
the debriefings. This includes debriefing the offeror’s 
strengths and weaknesses, giving the offeror his ratings, 
covering all evaluated areas and debriefing down to the sub- 
factor level and releasing the SSA decision document. 


Contracting officers should also use the latitude they have to 
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deviate from the strict guidance of the AFARS when it is in 


the best interests of the parties involved. 


E. AREAS FOR FURTHER RESEARCH 

An area for further research can be to look at the 
debriefing process by contractor commodity type, ie. hardware, 
software, services, and compare these commodity types to see 
if the contractors in each type face the same problems with 
debriefings or if debriefings in different commodities have 
different limitations. Another approach to take with further 
resea: 1 could be to research debriefings by large and small 
businesses to see if the two groups face the same or different 


problems with debriefings. 
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APPENDIX 


SURVEY FOR UNSUCCESSFUL OFFERORS CONCERNING ARMY DEBRIEFING 


1. When not awarded a contract I request a debriefing 
a. more than 2/3's of the time. 
b. between 1/3 and 2/3's of the time. 
c. less than 1/3 of the time. 

Comments: 


2. The Army gives a debrief every time I request one. 


a. yes 
be no 
Comments: 


3. The following personnel (by job title) from my company 
attend the debriefing. 


Comments: 


4. The Army limits the number of employees that may attend 
the debriefing from my company. 

a. always 

b. sometimes 

c. never 
Comments: 


5. The Army limits the employees, by job position, that may 
attend the debriefing from my company. 

a. always 

b. sometimes 

C. never 
Comments: 


YH 


6. Debriefings are normally held within calendar days of 
my request. 


a. 0 - 15 
Dine 
c. more than 20 
Comments: 
7. Debriefings should be conducted within calendar 
days after contract award. 
Comments: 
8- normally debriefs me. 
a. the contracting officer 
b. source selection evaluation board chairman 
c. a combination of both 
d. other 
Comments: 


9. Debriefings are normally held 
a. at my location. 
b. at the Army’s location. 
c. other location. 
d. by telephone 
Comments: 


10. The Army 
a. doesn’t allow questions. 
b. allows and answers all questions. 
c. allows and answers some questions. 


Comments: 
11. If the Army doesn't answer some of my questions it is 
because 

a. they can’t answer them due to regulations or 


confidential/proprietary business information. 

b. they won’t answer them due to their concern for a 
protest. 

c. other 

d. not applicable, they answer all my questions. 
Comments: 
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12. Debriefings normally consist of 
a. limited information, because 


b. aS much information as the Army can give in accordance 
with the regulations and confidential/proprietary business 
information 
Comments: 


13. The Army would provide more information if the possible 
threat of a protest did not exist. 

a. always 

b. sometimes 


c. never 
d. don't know 
Comments: 


14. The Army debriefs in accordance with the major elements of 
the RFP. 

a. always 

b. sometimes 

c. never 
Comments: 


15. Debriefings clearly identify weaknesses in my proposal. 
a. always 
b. sometimes 


c. never 
Comments: 
a6 . The Army debriefs me on the merits of my 


proposal (circle all that apply). 
a. technical 
b. management 
ee cost 

Comments: 


an]... I am satisfied with the debriefing on the technical, 
Management, and cost elements of my proposal. 

a. always 

b. sometimes 

c. never 
Comments: 
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coe Debriefings give me information that leads to more 
competitive proposals in future Government competition. 

a. always 

b. sometimes 

c. never 
Comments: 


19. Upon conclusion of a debrief, I completely understood why 
my company did not win the contract award. 

a. always 

b. sometimes 

c. never, why not 
Comments: 


20. The debriefings are to my company. 
a. valuable 
b. somewhat valuable 
c. not at all valuable 

Comments: 


21. The Army offers to debrief me even when I don’t ask for 
a debriefing. 

a. always 

b. sometimes 


c. never 
Comments: 
227 Debriefings should address (circle all that apply) 


, as long as confidential business information is 

not disclosed. 

a. the basic proposed technical solution of the awardee 

b. the overall evaluated cost of the awardee and debriefed 
offeror 

Cc. cost or price associated with the major components of 
the awardee's proposal 

d. overall ranking and total evaluation scores of the 
awardee and debriefed offeror 

e. technical point scores of the awardee and debriefed 
offeror 

f. none of the above 

g. other 
Comments: 
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23. The sooner I’m debriefed the less likely I am to protest. 
a. always 
b. sometimes 
c. never 

Comments: 


24. List 3 strengths of the Army debriefing process. 


25. List 3 weaknesses of the Army debriefing process. 


26. The 3 most common questions I ask during a debriefing. 


27. The Army could improve the debriefing process by 


28o: I/unsuccessful offerors could improve the debriefing 
process by 
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SURVEY FOR AMC CONTRACTING OFFICERS CONCERNING DEBRIEFINGS 


1. How many unsuccessful offerors ask for a debriefing? 
a. less than 1/3 
b. between 1/3 and 2/3 
c. more than 2/3 

Comments: 


2. I give a debriefing every time one is requested. 


a. yes 
bD. ic 
Comments: 
e The following personnel (by job title) from my 


organization attend the debriefing 


Comments: 


4. I limit the number of employees that unsuccessful offerors 
may send to a debriefing. 

a. always 

b. sometimes 


c. never 
Comments: 
Se I limit employees, by job title, that unsuccessful 


offerors may send to a debriefing. 
a. always 
b. sometimes 
c. never 

Comments: 


6. Debriefings are held within days of debriefing 
request. 

a. 0-10 

b. 11-20 

Cc. more than 20 
Comments: 


7.  Debriefings should be conducted within calendar 
days after contract award. 
Comments: 
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8. Debriefings are conducted by (Give job title) 


Comments: 


9. Debriefings are normally held 
a. at my location 
b. at the unsuccessful offeror’s location 
c. other location 
d. by telephone 
Comments: 


mon I 
a. don’t allow questions 
b. allow and answer all questions 
c. allow and answer some questions 
Comments: 


Al. If I don't answer some of the unsuccessful offerors 
questions it is because 
a. I can't answer them due to regulations or 
confidential/proprietary business information 
b. I won't answer them due to a possible protest 


C. other 
d. n/a, I answer all questions 
Comments: 


12. Debriefings normally consist of 
a. limited information, because 


b. aS much information as I can give in accordance with 
the regulations and confidential/proprietary business 
information 
Comments: 


13. I would provide more information if the possible threat 
of a protest did not exist. 

a. always 

b. sometimes 

c. never 
Comments: 
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14. 1 debrief in accordance with the major elements of the 
RFP. 

a. always 

b. sometimes 


c. never 
Comments: 
157 Debriefings clearly identify weaknesses in the 


unsuccessful offeror's proposal. 
a. always 
b. sometimes 


c. never 
Comments: 
16. I debrief unsuccessful offerors on the merits of 


their proposal (circle all that apply). 
a. technical 
b. management 
Ge COSE 

Comments: 


17. Unsuccessful offerors are satisfied with the debriefing 
on the technical, management, and cost elements of their 
proposal. 

a. always 

b. sometimes 


c. never 
ad. don’t know 
Comments: 


18. Debriefings give unsuccessful offerors information that 
leads to more competitive proposals for future Government 
competition. 

a. always 

b. sometimes 


c. never 
G. don’t know 
Comments: 
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19. Upon conclusion of a debrief the unsuccessful offeror 
completely understands why he didn't win the contract award. 
a. always 
b. sometimes 
c. never 
da. don’t know 
Comments: 


20 . The debriefings I give are to unsuccessful 
offerors. 

a. valuable 

b. somewhat valuable 

c. not at all valuable 
Comments: 


aL. My organization offers to debrief all unsuccessful 
offerors, even those that don't request a debriefing. 

a. always 

b. sometimes 

c. never 
Comments: 


22. Debriefings should address (circle all that apply) 
, as long as confidential business 
information is not disclosed. 
a. the basic proposed technical solution of the awardee 
b. the overall evaluated cost of the awardee and debriefed 
offeror 
Cc. cost or price associated with the major components of 
the awardee’s proposal 
d. overall ranking and total evaluation scores of the 
awardee and debriefed offeror 
e. technical point scores of the awardee and debriefed 
offeror 
f. none of the above 
g. other 
Comments: 





23. The sooner a debriefing is conducted, the more likely it 
is to prevent a protest. 

a. always 

b. sometimes 

c. never 
Comments: 
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24. My organization has internal instructions covering 
debriefing procedures 


a. yes 
b. no 
Comments: 


25. List 3 strengths of the Army debriefing process 


26. List 3 weaknesses of the Army debriefing process 


27. I/The Army could improve the debriefing process by 


28. Unsuccessful offerors could improve the debriefing by _ 
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